CheckpointManager 2017-04-25T13:00:49.177-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:00:49.197-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:00:49.197-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:00:49.154-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
2017/04/25 13:00:49 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:00:49 UPR_STREAMREQ for vb 1 successful
StatisticsManager 2017-04-25T13:00:50.152-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:00:50 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:00:50 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:00:51 UPR_STREAMREQ for vb 29 successful
StatisticsManager 2017-04-25T13:00:52.546-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:00:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:00:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:00:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:00:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:00:53 UPR_STREAMREQ for vb 133 successful
StatisticsManager 2017-04-25T13:00:53.761-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:00:53 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:00:53 UPR_STREAMREQ for vb 234 successful
PipelineManager 2017-04-25T13:00:54.094-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:00:54 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:00:54 UPR_STREAMREQ for vb 333 successful
StatisticsManager 2017-04-25T13:00:54.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:00:54 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:00:54 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:00:56 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:00:56 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:00:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 335 successful
GenericSupervisor 2017-04-25T13:00:58.071-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:00:58.302-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:00:58.306-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:00:36.110027369-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:00:58.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:00:58.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:00:58.306-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:00:58.306-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [145 164 51 97 141 163 166 168 169 95 98 150 143 146 167 56 86 81 41 54 72 37 53 82 152 50 55 153 87 156 40 73 96 139 77 99 38 70 75 76 84 142 157 65 90 138 63 83 39 44 60 147 151 43 170 57 93 155 47 91 144 162 62 67 71 137 161 165 66 74 85 79 94 36 42 45 136 149 89 140 46 59 52 80 148 159 48 58 61 64 69 49 88 92 78 154 160 68 158]
StatisticsManager 2017-04-25T13:00:58.306-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [273 340 188 249 267 275 253 283 294 189 262 287 171 241 247 251 276 292 192 172 240 242 261 280 284 190 290 293 250 285 338 244 334 185 281 274 298 196 265 270 272 186 238 268 288 297 239 286 182 279 282 243 258 337 289 255 264 199 269 180 271 257 198 173 177 183 184 245 335 339 174 175 254 178 248 252 266 278 296 336 197 176 195 237 256 291 181 246 179 193 259 277 295 299 194 260 263 187 191]
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:00:58.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
2017/04/25 13:00:58 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:00:58 UPR_STREAMREQ for vb 45 successful
ToplogyChangeDetector 2017-04-25T13:00:59.045-05:00 [INFO] validateSourceTopology...
2017/04/25 13:00:59 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 246 successful
StatisticsManager 2017-04-25T13:00:59.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:00:59.114-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:00:59.118-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:00:59.118-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:00:59 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:00:59 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:01:00 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:01:00 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:01:00 UPR_STREAMREQ for vb 253 successful
StatisticsManager 2017-04-25T13:01:00.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:01 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 261 successful
StatisticsManager 2017-04-25T13:01:01.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:01 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:01:01 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:01:02 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:01:03 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:01:03 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:01:03 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:01:07 UPR_STREAMREQ for vb 67 successful
StatisticsManager 2017-04-25T13:01:03.114-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:08 UPR_STREAMREQ for vb 270 successful
GenericSupervisor 2017-04-25T13:01:08.215-05:00 [INFO] About to call function
2017/04/25 13:01:08 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:01:08 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:01:08 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:01:08 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:01:09 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:01:09 UPR_STREAMREQ for vb 274 successful
ToplogyChangeDetector 2017-04-25T13:01:09.742-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:01:09.783-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:01:09.783-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:01:09.783-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:01:09 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:01:09 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:01:09 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:01:09 UPR_STREAMREQ for vb 276 successful
StatisticsManager 2017-04-25T13:01:09.938-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:11 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 73 successful
DcpNozzle 2017-04-25T13:01:11.673-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
2017/04/25 13:01:11 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 278 successful
StatisticsManager 2017-04-25T13:01:11.744-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:01:11.747-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:00:36.110027369-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [284 190 280 290 293 285 185 281 298 196 186 288 297 286 182 279 282 289 199 180 198 173 177 174 175 183 184 178 197 278 296 176 195 181 291 179 193 194 295 299 187 191 188 189 283 294 287 192 292]
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [90 83 93 91 74 85 79 94 89 80 88 92 78 97 95 98 86 81 82 87 96 77 84 99 75 76]
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
2017/04/25 13:01:11 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 279 successful
StatisticsManager 2017-04-25T13:01:11.848-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:11 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:01:11 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:01:12 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:01:12 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:01:12 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:01:12 UPR_STREAMREQ for vb 177 successful
StatisticsManager 2017-04-25T13:01:12.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:01:12.126-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:01:12.189-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:01:15 UPR_STREAMREQ for vb 80 successful
StatisticsManager 2017-04-25T13:01:15.689-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:01:15.894-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:01:15.894-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:01:15.935-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:15 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:01:15 UPR_STREAMREQ for vb 186 successful
CapiNozzle 2017-04-25T13:01:15.991-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:01:15.995-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:01:16 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:01:16 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:01:16 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:01:16 UPR_STREAMREQ for vb 188 successful
CapiNozzle 2017-04-25T13:01:16.060-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

2017/04/25 13:01:16 UPR_STREAMREQ for vb 87 successful
CapiNozzle 2017-04-25T13:01:16.109-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
StatisticsManager 2017-04-25T13:01:16.858-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:01:16.886-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:01:16.923-05:00 [INFO] Send batch count=318
CapiNozzle 2017-04-25T13:01:17.178-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:01:17.208-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CheckpointManager 2017-04-25T13:01:17.242-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895
CheckpointManager 2017-04-25T13:01:17.243-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:01:17 UPR_STREAMREQ for vb 89 successful
DcpNozzle 2017-04-25T13:01:17.321-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:01:17 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:01:17 UPR_STREAMREQ for vb 99 successful
StatisticsManager 2017-04-25T13:01:17.396-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:01:17.420-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:01:17.503-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:01:17.521-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
GenericSupervisor 2017-04-25T13:01:17.819-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:01:17.827-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:00:36.110027369-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 281, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 299, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 3106, "docs_rep_queue": 2605, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 319, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 299, "set_filtered": 0, "set_received_from_dcp": 3106, "size_rep_queue": 3359345, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 2638}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 299, "docs_opt_repd": 0, "docs_rep_queue": 2287, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 299, "size_rep_queue": 2929848, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 409, "max": 2642, "mean": 2638.1026894865527, "min": 2633}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 252, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 154, "docs_opt_repd": 1, "docs_rep_queue": 523, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 1, "set_failed_cr_source": 154, "size_rep_queue": 903066, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 2787, "max": 34874, "mean": 51.054, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 4639, "expiry_received_from_dcp": 0, "set_received_from_dcp": 4639}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 319, "max": 725373, "mean": 2290.282131661442, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 1864, "expiry_received_from_dcp": 0, "set_received_from_dcp": 1864}}

StatisticsManager 2017-04-25T13:01:17.827-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 4639 items, sent 500 items
StatisticsManager 2017-04-25T13:01:17.827-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 1864 items, sent 319 items
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:4639]
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 4639 items, sent 4639 items.
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:1864]
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 1864 items, sent 1864 items.
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:01:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
CapiNozzle 2017-04-25T13:01:17.853-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:01:17.871-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
StatisticsManager 2017-04-25T13:01:17.957-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
ToplogyChangeDetector 2017-04-25T13:01:20.238-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:01:23.114-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:01:23.056-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:01:23.349-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:01:20.172-05:00 [INFO] Executing 0x681fb0 timed out
CapiNozzle 2017-04-25T13:01:25.144-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:01:25.244-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:01:25.244-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
GenericSupervisor 2017-04-25T13:01:25.268-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:01:25.268-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:01:25.310-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:01:25.310-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:01:25.316-05:00 [INFO] err_list=[{"time":"2017-04-25T13:01:25.316123792-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:01:25.316-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:01:25.353-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:01:25.818-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
StatisticsManager 2017-04-25T13:01:28.685-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:01:28.685-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895 {"Errors": "[{\"time\":\"2017-04-25T13:01:25.316123792-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 335, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 453, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 7383, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 453, "set_filtered": 0, "set_received_from_dcp": 7383, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:01:28.709-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: i/o timeout
CapiNozzle 2017-04-25T13:01:28.710-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: i/o timeout.
GenericSupervisor 2017-04-25T13:01:28.710-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:01:28.710-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:01:28.710-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:01:28.761-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:01:28.761-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:01:28.831-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:01:28.888-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
ToplogyChangeDetector 2017-04-25T13:01:29.415-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:01:30.573-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:01:25.316123792-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:01:32.737-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:01:32.664-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:01:33.150-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:01:33.550-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:01:34.223-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:01:34.267-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:01:34.409-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
PipelineManager 2017-04-25T13:01:35.471-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:01:37.075-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:01:37.168-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:01:37.334-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895
PipelineRuntimeCtx 2017-04-25T13:01:37.334-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:01:37.334-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:01:37.645-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:01:37.627-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:01:37.334340523 -0500 CDT
GenericSupervisor 2017-04-25T13:01:39.617-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:01:40.173-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:01:40.584-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:01:40.595-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CheckpointManager 2017-04-25T13:01:40.556-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:01:40.666-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:01:40.963-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:01:41.335-05:00 [INFO] StatisticsManager Stopped
GenericSupervisor 2017-04-25T13:01:41.590-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
ToplogyChangeDetector 2017-04-25T13:01:42.756-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:01:42.737-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
PipelineRuntimeCtx 2017-04-25T13:01:42.876-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-713429895 has been stopped
GenericSupervisor 2017-04-25T13:01:43.457-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:01:46.583-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
GenericPipeline 2017-04-25T13:01:46.738-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-713429895
GenericPipeline 2017-04-25T13:01:46.748-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-713429895
GenericPipeline 2017-04-25T13:01:46.748-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-713429895
GenericPipeline 2017-04-25T13:01:46.748-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-713429895
AsyncListener 2017-04-25T13:01:46.749-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:01:46.758-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:01:46.758-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:01:46.758-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:01:46.758-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:01:46.758-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:01:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:01:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:01:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:01:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:01:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:01:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:01:46.759-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:01:46.762-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
GenericSupervisor 2017-04-25T13:01:45.515-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:01:46.808-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:01:46.811-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:01:46.816-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:01:46.820-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:01:46.820-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:01:46.820-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:01:46.832-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:01:46.832-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:01:46.832-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:01:46.832-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.838-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.838-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.838-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.838-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.838-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.839-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.839-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.839-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.839-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.839-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:01:46.839-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:01:46.862-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
GenericSupervisor 2017-04-25T13:01:46.868-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:01:46.868-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:01:46.832633663 -0500 CDT
AsyncListener 2017-04-25T13:01:47.185-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.059-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.125-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.240-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.156-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.263-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:01:47.083-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.797-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:01:47.308-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.824-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:01:47.210-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:01:46.844-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:01:46.816122165 -0500 CDT
AsyncListener 2017-04-25T13:01:47.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:01:46.770-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:01:47.923-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:01:47.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:01:47.951-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:01:47.951-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:01:47.747-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:01:46.839-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:01:47.308-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:01:47.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:48.056-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:01:47.348-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:01:47.348-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:01:48.524-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:01:49.824-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:01:49.824-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:01:49.824-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:01:49.826-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-713429895 is stopped
PipelineManager 2017-04-25T13:01:49.826-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:01:49.826-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:01:25.316123792-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:01:49.826-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:01:48.180-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:01:48.464-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:01:49.827-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:01:49.827-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:01:49.827-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:01:48.409-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:01:49.827-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:01:49.827-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:01:49.827-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
AsyncListener 2017-04-25T13:01:48.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericSupervisor 2017-04-25T13:01:49.869-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:01:49.477-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:01:49.870-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:01:49.870-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:01:49.870-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericSupervisor 2017-04-25T13:01:49.870-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:01:49.871-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:01:49.871-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:01:49.871-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
DcpNozzle 2017-04-25T13:01:49.871-05:00 [INFO] Ask uprfeed to close
GenericSupervisor 2017-04-25T13:01:49.871-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:01:49.871-05:00 [INFO] releasing capi client
DcpNozzle 2017-04-25T13:01:49.871-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:01:49.871938984 -0500 CDT
DcpNozzle 2017-04-25T13:01:49.872-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:01:49.872-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:01:49.872-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:01:49.872195298 -0500 CDT
DcpNozzle 2017-04-25T13:01:49.872-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:01:49 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:01:49 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:01:49 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:01:49 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:01:49 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:01:49 Stream Closed for vb 4 StreamEnd simulated
2017/04/25 13:01:49 Stream Closed for vb 5 StreamEnd simulated
PipelineManager 2017-04-25T13:01:49.962-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:01:25.316123792-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:01:51.389-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:01:51.389-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:01:51.389-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:01:51.434-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:01:51.435-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:01:51.435-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:01:51.435-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:01:51.435-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:01:51.435-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:01:51.435-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:01:51.435-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:01:51.436-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
GenericSupervisor 2017-04-25T13:01:51.437-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:01:51.437-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
GenericSupervisor 2017-04-25T13:01:51.437-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:01:51.437-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:01:51.437-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:01:51.437-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:01:51.437-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:01:51.438-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
CheckpointManager 2017-04-25T13:01:51.438-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
CapiUtils 2017-04-25T13:01:51.461-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:01:51.461-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:01:51.461-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:01:51.462-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
StatisticsManager 2017-04-25T13:01:51.462-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:01:51.462-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
XDCRFactory 2017-04-25T13:01:51.462-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:01:51.462-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443 is constructed, start it
GenericPipeline 2017-04-25T13:01:51.463-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[16 52 80 81 101 140 32 75 90 109 14 147 42 151 3 12 21 35 44 76 114 130 59 67 72 95 127 53 106 159 166 169 28 55 78 167 19 48 162 164 30 65 137 143 49 100 105 154 17 115 146 86 117 33 89 156 23 63 120 111 134 34 58 70 79 40 113 123 29 82 138 170 133 158 25 41 149 5 38 93 107 153 62 160 163 39 61 85 1 26 119 157 37 142 50 60 148 69 77 168 145 0 57 112 144 128 152 7 13 91 121 155 102 108 54 64 71 94 18 36 97 150 6 15 68 51 73 88 131 99 22 125 126 129 31 165 27 84 2 98 103 135 45 47 56 74 132 66 92 10 43 9 116 124 136 122 24 83 161 46 96 139 141 4 11 20 118 87 104 110 8] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[209 335 243 220 318 325 314 225 255 197 221 262 327 171 188 296 300 308 334 180 277 205 226 332 187 306 330 198 261 307 338 242 303 216 309 193 214 244 239 248 258 329 179 181 211 331 222 272 312 263 231 227 238 223 250 202 337 194 245 256 269 311 237 178 201 289 229 276 190 264 270 310 186 235 265 271 273 292 293 319 210 313 320 304 174 189 200 286 282 199 234 213 252 288 291 183 280 326 246 268 305 176 191 240 257 275 285 316 184 215 219 260 230 302 185 232 323 266 301 207 224 253 290 322 195 249 267 281 287 339 340 192 241 321 175 177 206 298 218 233 247 294 336 212 279 297 196 208 328 333 172 204 251 182 274 295 284 259 283 299 315 228 173 317 324 203 217 254 278 236]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[310 186 235 265 271 319 210 273 292 293 304 313 320 286 174 189 200 199 234 282 291 183 213 252 288 246 280 326 268 305 191 176 240 257 184 215 219 275 285 316 260 230 302 185 232 323 266 301 290 207 224 253 195 249 267 322 281 287 192 241 339 340 175 177 321 206 298 233 218 247 294 336 196 208 212 279 297 172 204 328 333 251 182 274 295 284 315 228 259 283 299 173 317 324 203 217 254 278 236 209 335 243 325 220 318 225 255 314 221 197 262 327 171 188 296 180 300 308 334 277 205 226 332 187 306 330 198 261 307 242 303 338 193 214 216 309 244 258 239 248 329 179 181 211 331 222 272 312 231 263 227 238 223 250 202 194 337 256 245 269 311 178 237 201 289 229 276 190 264 270] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[41 149 5 38 93 107 153 62 160 163 39 61 85 1 26 119 157 37 142 50 60 148 69 77 168 0 57 112 144 145 7 13 91 121 128 152 155 54 64 71 94 102 108 18 36 97 150 6 15 68 51 73 88 131 99 22 125 126 129 31 165 27 84 2 98 45 47 56 74 103 135 132 66 92 10 43 9 116 124 136 122 24 83 161 46 96 139 141 4 11 20 118 87 104 110 8 16 52 80 81 101 32 75 90 109 140 14 147 3 12 21 35 42 151 44 76 114 130 59 67 72 95 127 53 106 159 166 169 28 55 78 19 48 162 164 167 30 65 137 143 49 100 105 154 17 115 146 86 117 33 89 156 23 63 120 34 58 70 79 111 134 40 113 123 29 82 138 170 133 158 25]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 replication_type:capi filter_expression: source_nozzle_per_node:2 log_level:Info target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:01:51.463-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443...
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:01:51.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
StatisticsManager 2017-04-25T13:01:51.471-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:01:51.486-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:01:51.486-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:01:51.486-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:01:51.491-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:01:51.491-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:01:51.491-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:01:51.491-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:01:51.491-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:01:51.491-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:01:51.491-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:01:51.491-05:00 [INFO] Checkpointing starts in 1801.21 sec
PipelineRuntimeCtx 2017-04-25T13:01:51.491-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:01:51.491-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:01:51.492-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:01:51.492-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:01:51.494-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:01:51.535-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:01:51.535-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:01:51.535-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:01:51.554-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:01:51.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
ToplogyChangeDetector 2017-04-25T13:01:51.556-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:01:51.557-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:01:51.595-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ReplicationManager 2017-04-25T13:01:51.601-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
DcpNozzle 2017-04-25T13:01:51.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:01:51.834-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:01:51.834-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:01:51.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:01:51.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:01:51.835-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:01:51.857-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:01:51.857-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:01:51.857-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:01:51.897-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:01:51.897-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:01:52.381-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:01:52.480-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:01:52.480-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:01:52.480-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:01:52.481-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:01:52.481-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:01:52.481-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:01:52.481-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443 is started----------
PipelineManager 2017-04-25T13:01:52.481-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:01:52.481-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:01:52.481-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:01:52.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:01:52.604-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CheckpointManager 2017-04-25T13:01:52.704-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:01:52.718-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:01:52.821-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:01:53.010-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
2017/04/25 13:01:53 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 206 successful
StatisticsManager 2017-04-25T13:01:53.868-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:53 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:01:53 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:01:54 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:01:55 UPR_STREAMREQ for vb 333 successful
StatisticsManager 2017-04-25T13:01:56.405-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:56 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 137 successful
StatisticsManager 2017-04-25T13:01:56.573-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:56 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 232 successful
StatisticsManager 2017-04-25T13:01:56.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:56 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:01:56 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 81 successful
StatisticsManager 2017-04-25T13:01:57.655-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:01:57 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:01:57 UPR_STREAMREQ for vb 189 successful
CapiNozzle 2017-04-25T13:01:58.074-05:00 [INFO] Send batch count=500
2017/04/25 13:01:58 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 97 successful
DcpNozzle 2017-04-25T13:01:58.324-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:01:58 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 294 successful
CapiNozzle 2017-04-25T13:01:58.381-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:01:58.442-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
DcpNozzle 2017-04-25T13:01:58.680-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:01:58 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:01:58 UPR_STREAMREQ for vb 299 successful
CheckpointManager 2017-04-25T13:01:58.640-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
CheckpointManager 2017-04-25T13:01:58.712-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:01:58.735-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
StatisticsManager 2017-04-25T13:01:58.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:01:59.963-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:02:00.002-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
StatisticsManager 2017-04-25T13:02:00.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:02:00.654-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:02:00.838-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:02:00.850-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:02:01.256-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:02:01.256-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:02:01.256-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:02:01.256-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
ReplicationManager 2017-04-25T13:02:01.256-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
CapiNozzle 2017-04-25T13:02:01.467-05:00 [INFO] Send batch count=33
StatisticsManager 2017-04-25T13:02:01.527-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:02:01.528-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 293, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 2781, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 293, "set_filtered": 0, "set_received_from_dcp": 2781, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

GenericSupervisor 2017-04-25T13:02:01.528-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:02:01.528-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:02:01.528-05:00 [INFO] Finish executing 0x681fb0
PipelineManager 2017-04-25T13:02:01.619-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:02:01.619-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:02:01.619-05:00 [INFO] err_list=[{"time":"2017-04-25T13:02:01.619933352-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:02:01.620-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:02:01.712-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:01.718-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:02:01.734-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:02:02.121-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:02:02.135-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:02:02.592-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:02:02.606-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:02:02.982-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:02:02.982-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:02:02.982-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:02:02.982-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:02:02.982-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:02:03.038-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:03.038-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:02:03.058-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:02:03.337-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:02:03.350-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
ToplogyChangeDetector 2017-04-25T13:02:03.449-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:02:03.584-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:02:03.601-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:02:05.538-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:02:06.285-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:02:06.285-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:02:06.285-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:02:06.286-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:02:06.701-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:06.908-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:02:06.654-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:02:01.619933352-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]}
]
CapiNozzle 2017-04-25T13:02:06.939-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:02:07.451-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:02:07.504-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:02:08.899-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:02:08.910-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:02:09.718-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:02:09.814-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:02:09.830-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:02:09.864-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:02:09.994-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:02:11.453-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:11.453-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:02:11.464-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:02:11.542-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:02:11.555-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
PipelineManager 2017-04-25T13:02:11.626-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
PipelineManager 2017-04-25T13:02:11.632-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:02:11.639-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:02:11.655-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
PipelineRuntimeCtx 2017-04-25T13:02:11.655-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:02:11.655-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:02:11.655-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:02:11.658-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:02:11.655533857 -0500 CDT
GenericSupervisor 2017-04-25T13:02:11.667-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:02:11.671-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:02:11.672-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:02:11.672-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:02:11.672-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:02:11.672-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:02:11.672-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:02:11.683-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:02:11.706-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:02:11.706-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-435606443 has been stopped
GenericPipeline 2017-04-25T13:02:11.716-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-435606443 is stopped
PipelineManager 2017-04-25T13:02:11.716-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:02:11.716-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:02:01.619933352-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:02:11.716-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:02:11.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:02:11.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:02:11.733-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:02:11.742-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:02:11.742-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:02:11.743-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
AsyncListener 2017-04-25T13:02:11.742-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:02:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:02:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:02:11.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:02:11.749-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
GenericPipeline 2017-04-25T13:02:11.749-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
GenericPipeline 2017-04-25T13:02:11.749-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-435606443
AsyncListener 2017-04-25T13:02:11.749-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:02:11.749-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:02:11.749-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:02:11.749-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:02:11.749-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:02:11.749-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:02:11.749-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:02:11.751-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:02:11.751-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:02:11.751-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:02:11.751-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:02:11.751-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:02:11.751-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:02:11.751-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:02:11.751-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:02:11.751-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:02:11.751-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.751-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.751-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:02:11.751-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:02:11.751377891 -0500 CDT
DcpNozzle 2017-04-25T13:02:11.751-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:02:11.752-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:02:11.751715598 -0500 CDT
AsyncListener 2017-04-25T13:02:11.752-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.752-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.752-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.752-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:02:11.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:02:11.754-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:02:11.754-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:02:11.754-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:02:11.754-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:02:11.754-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:02:11.754-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:02:11.755-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:02:11.755-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:02:11.755868895 -0500 CDT
DcpNozzle 2017-04-25T13:02:11.755-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:02:11.755-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:02:11.756-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
AsyncListener 2017-04-25T13:02:11.756-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.756-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:02:11.756-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.756-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:02:11.756-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:02:11.756-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:02:11.758-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:02:11.758-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:02:11.75812429 -0500 CDT
DcpNozzle 2017-04-25T13:02:11.758-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:02:11.758-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:02:11.758-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:02:11.758-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:02:11.758-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:02:11.758-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:02:11.759-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
2017/04/25 13:02:11 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:02:11 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:02:11 Stream Closed for vb 1 StreamEnd simulated
XDCRFactory 2017-04-25T13:02:15.785-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:02:15.786-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:02:15.787-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:02:15.877-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:02:15.963-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:02:15.998-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:02:15.998-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:02:15.998-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:02:15.998-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:02:16.014-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:02:16.051-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:02:16.104-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
GenericSupervisor 2017-04-25T13:02:16.118-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:02:16.120-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
GenericSupervisor 2017-04-25T13:02:16.120-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:02:16.120-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:02:16.120-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:02:16.120-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:02:16.120-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:02:16.121-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
CheckpointManager 2017-04-25T13:02:16.121-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
CapiUtils 2017-04-25T13:02:16.208-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:02:16.228-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:02:16.257-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:02:16.506-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
StatisticsManager 2017-04-25T13:02:16.554-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:02:16.554-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
XDCRFactory 2017-04-25T13:02:16.554-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:02:16.555-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 is constructed, start it
GenericPipeline 2017-04-25T13:02:16.557-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[208 221 222 200 231 179 197 247 242 265 229 172 199 220 228 284 263 176 276 321 335 223 243 252 323 337 300 304 318 235 251 217 289 291 313 326 230 239 193 214 330 336 317 329 182 195 210 232 236 316 307 294 206 273 280 312 328 260 305 233 270 198 224 177 297 322 298 314 173 185 332 183 261 334 226 254 253 301 320 174 194 225 212 244 282 309 259 340 246 205 274 288 311 203 181 192 293 256 333 257 331 296 266 272 339 213 245 178 202 319 248 204 237 327 175 238 299 264 268 286 191 277 290 216 325 188 219 306 201 255 295 196 278 186 292 215 240 279 283 303 338 234 271 287 315 171 184 209 187 189 218 227 241 308 262 275 285 190 211 250 269 281 324 207 267 180 258 310 249 302] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[19 75 92 9 113 48 54 165 118 2 65 117 32 41 153 10 18 31 94 138 150 132 133 35 79 106 53 146 3 21 29 157 50 96 147 141 39 57 130 6 52 73 82 68 143 5 135 156 25 86 110 43 74 76 69 108 168 28 125 91 103 16 26 58 119 93 97 111 98 8 34 63 64 101 120 51 169 15 78 83 129 7 42 59 4 124 33 140 145 1 47 136 22 123 160 163 13 71 159 38 131 62 144 81 149 37 0 115 166 36 89 66 60 170 158 121 127 142 14 87 128 23 27 95 90 102 20 44 17 148 164 151 155 61 112 139 46 88 99 56 109 137 122 30 84 114 85 40 67 72 12 55 154 100 45 116 152 49 134 167 11 162 77 104 105 107 126 24 70 80 161]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[78 83 169 15 42 59 129 7 124 4 140 145 33 47 136 1 123 160 22 71 159 163 13 131 38 144 62 149 81 37 115 0 166 89 36 66 170 60 127 142 158 121 87 128 14 27 95 23 102 90 44 20 148 164 17 112 139 151 155 61 88 99 46 109 137 56 84 114 122 30 67 72 85 40 55 154 12 100 116 152 45 134 167 49 162 11 77 70 80 104 105 107 126 24 161 75 92 19 113 9 54 165 48 118 65 117 2 18 31 32 41 153 10 138 150 94 79 106 132 133 35 21 29 53 146 3 96 147 157 50 57 130 141 39 6 73 52 82 143 68 135 156 5 25 74 76 86 110 43 108 168 69 28 125 26 58 91 103 16 97 111 119 93 34 63 98 8 101 120 64 51] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[332 183 261 334 254 226 174 253 301 320 194 225 212 244 282 309 259 340 246 203 205 274 288 311 181 192 256 293 257 331 333 266 296 213 245 272 339 178 202 319 248 237 327 204 175 238 299 191 264 268 286 216 277 290 219 306 325 188 201 255 295 196 278 186 292 215 240 279 283 303 338 234 271 287 315 171 184 209 187 189 218 227 241 308 190 211 262 275 285 250 269 281 324 180 207 267 258 310 249 302 208 221 222 179 200 231 197 247 242 265 229 172 199 228 284 220 263 176 276 321 335 223 243 252 304 318 323 337 300 235 251 217 289 291 313 326 193 214 230 239 317 330 336 182 195 329 210 232 236 316 307 294 206 273 280 312 260 305 328 198 224 233 270 177 297 322 298 314 173 185]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 optimistic_replication_threshold:256 log_level:Info stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:02:16.651-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886...
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:02:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:02:16.671-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:02:16.671-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:02:16.671-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:02:16.672-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:02:16.671-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:02:16.673-05:00 [INFO] Checkpointing starts in 1803.937 sec
PipelineRuntimeCtx 2017-04-25T13:02:16.673-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:02:16.675-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:02:16.676-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:02:16.684-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:02:16.684-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:02:16.684-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:02:16.697-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:02:16.708-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:02:16.708-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:02:16.708-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:02:16.708-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:02:16.713-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:02:16.774-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:02:16.801-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
StatisticsManager 2017-04-25T13:02:16.816-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:02:16.859-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CheckpointManager 2017-04-25T13:02:16.919-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:02:16.919-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:02:16.919-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiNozzle 2017-04-25T13:02:16.860-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:02:17.890-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:02:17.893-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:02:17.894-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
CheckpointManager 2017-04-25T13:02:17.901-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:02:17.901-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:02:17.901-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:02:17.901-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:02:17.902-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
DcpNozzle 2017-04-25T13:02:17.902-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
CapiNozzle 2017-04-25T13:02:17.902-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:02:17.929-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:02:17.929-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:02:17.929-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:02:17.953-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:02:17.953-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:02:17.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:02:18.110-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:02:18.110-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:02:18.110-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:02:18.110-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:02:18.110-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:02:18.110-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:02:18.110-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 is started----------
PipelineManager 2017-04-25T13:02:18.110-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:02:18.110-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:02:18.110-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:02:18 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 324 successful
StatisticsManager 2017-04-25T13:02:18.821-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:02:18 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:02:18 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 262 successful
CapiNozzle 2017-04-25T13:02:19.570-05:00 [INFO] Send batch count=328
2017/04/25 13:02:19 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 266 successful
CapiNozzle 2017-04-25T13:02:19.772-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:19.773-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:02:19.784-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:02:19 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 275 successful
StatisticsManager 2017-04-25T13:02:19.857-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:02:19 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:02:19 UPR_STREAMREQ for vb 183 successful
DcpNozzle 2017-04-25T13:02:20.002-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CheckpointManager 2017-04-25T13:02:20.019-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
CheckpointManager 2017-04-25T13:02:20.019-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:02:20 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:02:20 UPR_STREAMREQ for vb 194 successful
DcpNozzle 2017-04-25T13:02:20.126-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:02:20.869-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:02:20.962-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
PipelineManager 2017-04-25T13:02:21.793-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
StatisticsManager 2017-04-25T13:02:21.942-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:02:22.020-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:02:22.048-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:02:22.054-05:00 [INFO] Send batch count=54
CapiNozzle 2017-04-25T13:02:22.076-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:22.076-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:02:22.102-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
StatisticsManager 2017-04-25T13:02:54.342-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:02:01.619933352-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 437, "dcp_dispatch_time": 905, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 168, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 4652, "docs_rep_queue": 4324, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 927, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 168, "set_filtered": 0, "set_received_from_dcp": 4652, "size_rep_queue": 5967084, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 157}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 168, "docs_opt_repd": 0, "docs_rep_queue": 4324, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 168, "size_rep_queue": 5967084, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 269, "max": 162, "mean": 157.635687732342, "min": 156}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 25, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 25, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 4652, "max": 583156, "mean": 905.268, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 4652, "expiry_received_from_dcp": 0, "set_received_from_dcp": 4652}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 838, "expiry_received_from_dcp": 0, "set_received_from_dcp": 838}}

StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 4652 items, sent 328 items
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 926 items, sent 55 items
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:926]
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 926 items, sent 926 items.
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:4652]
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 4652 items, sent 4652 items.
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =88 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =87 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:02:54.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
ToplogyChangeDetector 2017-04-25T13:02:54.345-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:02:54.374-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:02:54.392-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:02:54.410-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:02:54.483-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:02:54.503-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:02:54.779-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:02:54.942-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
StatisticsManager 2017-04-25T13:02:55.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
StatisticsManager 2017-04-25T13:02:55.195-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:02:55.289-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:02:55.289-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:02:55.289-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:02:55.289-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
ReplicationManager 2017-04-25T13:02:55.289-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
PipelineManager 2017-04-25T13:02:55.309-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:02:55.310-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:02:55.312-05:00 [INFO] err_list=[{"time":"2017-04-25T13:02:55.312903622-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]
PipelineManager 2017-04-25T13:02:55.313-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
GenericSupervisor 2017-04-25T13:02:55.391-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:02:55.391-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:02:55.391-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:02:55.729-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:02:55.729-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:02:55.729-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:02:55.729-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:02:55.729-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:02:56.086-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:02:55.312903622-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]}
]
StatisticsManager 2017-04-25T13:02:56.422-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:02:56.423-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 {"Errors": "[{\"time\":\"2017-04-25T13:02:55.312903622-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10121, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 193, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 8797, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 193, "set_filtered": 0, "set_received_from_dcp": 8797, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]", "Status": "Pending"}

GenericSupervisor 2017-04-25T13:02:56.964-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:02:56.964-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:02:56.964-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:02:58.315-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:02:58.804-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:02:58.804-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:02:59.288-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:03:05.427-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:06.275-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:03:06.325-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries 
PipelineManager 2017-04-25T13:03:06.325-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:03:06.325-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:03:06.325-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
PipelineRuntimeCtx 2017-04-25T13:03:06.325-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:03:06.325-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:03:06.325-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:03:06.325-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:03:06.325-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:06.330-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:03:06.330-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:06.326025688 -0500 CDT
GenericSupervisor 2017-04-25T13:03:06.330-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:06.330-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:03:06.330-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:03:06.330-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:03:06.330-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:03:06.330-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:03:06.330-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 has been stopped
GenericPipeline 2017-04-25T13:03:06.330-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-7458886 is stopped
PipelineManager 2017-04-25T13:03:06.330-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:03:06.331-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:02:55.312903622-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:03:06.331-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:03:06.331-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
GenericPipeline 2017-04-25T13:03:06.331-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
GenericPipeline 2017-04-25T13:03:06.331-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
GenericPipeline 2017-04-25T13:03:06.331-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-7458886
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:06.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:06.332-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:06.332-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:06.332-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:03:06.332-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:06.332-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:03:06.332-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:03:06.332-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:03:06.332-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:03:06.332-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:03:06.332-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:06.332-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:03:06.332-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:03:06.333-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:03:06.333-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:06.333-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:06.333-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:03:06.333-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:06.332868223 -0500 CDT
CapiNozzle 2017-04-25T13:03:06.333-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:06.333173468 -0500 CDT
CapiNozzle 2017-04-25T13:03:06.363-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:03:06.363-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:03:06.363-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:03:06.363-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:06.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:06.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:03:06.364-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:06.364-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
panic: send on closed channel

goroutine 4135 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc3ca0ef440, 0x8edb60, 0xc265648a00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc3ca2ef050, 0x8ed920, 0xc311c526e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3ca005380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 4 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 4 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc20802b960)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080e6001, 0x5ff, 0x5ff, 0xc20800bc80, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080e6001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080e0000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e0000, 0x8f07a0, 0xc2080e2020, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc2080e2000, 0xc20801fbe0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc2080da000, 0x7f94f563be00, 0xc2080e2000, 0x0, 0x0, 0xc20801fbe0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc2080da000, 0x7f94f563be00, 0xc2080e2000, 0xc2080dc000, 0xc2080d8060, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc2080da000, 0x7f94f563be00, 0xc2080e2000)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b9f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b9f90, 0xc2080be480, 0x7f94f563a588, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 16 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc208084540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 12 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc208076840)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 13 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc208076840)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 15 [syscall, 4 minutes]:
syscall.Syscall(0x0, 0x0, 0xc20800f000, 0x1000, 0xc20803b1d0, 0xc2080326e0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc20800f000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc2080d5630, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc2080e2900)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc2080e2900, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 4391 [select]:
net/http.(*persistConn).writeLoop(0xc2f98e8e70)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4390 [IO wait]:
net.(*pollDesc).Wait(0xc2506247d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2506247d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc250624770, 0xc273532000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc210eb7900)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc34f5cc1e0, 0xc273532000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc34f5cc1e0, 0xc2f98e8ec8, 0xc273532000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2d4a3e340, 0xc273532000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2c2802b40)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2c2802b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2f98e8e70)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 33 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208084550)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 34 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 35 [select, 4 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 36 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc208080020, 0x11, 0xc208145fc0, 0xc2080e24e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc208080020, 0x11, 0xc208145fc0, 0xc2080e24e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc208080020, 0x11, 0xc208145fc0, 0xc2080e24e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080d4ff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 37 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080d1910, 0xf, 0xc208144fc0, 0xc2080e24e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080d1910, 0xf, 0xc208144fc0, 0xc2080e24e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080d1910, 0xf, 0xc208144fc0, 0xc2080e24e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080d5180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 38 [select]:
net/http.(*persistConn).roundTrip(0xc208076420, 0xc2f8e7b840, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc262abd450, 0xc2667c65f0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc262abd450, 0x7f94f5639fe8, 0xe94600, 0x49, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc262abd450, 0x49, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc262abd450, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc262abd450, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc262abd450, 0x7f94f563db20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc2e2870d20, 0xc263038280, 0x34, 0x7f94f563db20, 0xc263579760, 0x958760, 0xc31a6e5e60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc279677800, 0xc263038280, 0x34, 0x958760, 0xc31a6e5e60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Client).GetPool(0xc279677800, 0x9cae60, 0x7, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:767 +0x339
github.com/couchbase/goxdcr/utils.LocalPool(0xc2fa977cc0, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:89 +0x4a7
github.com/couchbase/goxdcr/utils.LocalBucketUUID(0xc2fa977cc0, 0xe, 0xc2080d1570, 0x8, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:370 +0x73
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080f41c0, 0xc3a5edf000, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:464 +0x12d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080f41c0, 0xc3a5edf000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc2080f48c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 39 [select]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc2080e28a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:102 +0x75b
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 40 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080e28b8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 24 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc208010760, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010760, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010700, 0xc2080ef000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc20811c490)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380d0, 0xc2080ef000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc2080380d0, 0xc208082058, 0xc2080ef000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080fa240, 0xc2080ef000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208066a80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208066a80, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208066a80, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bef60)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bef60, 0xc208164000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080daa40, 0xc208164000, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080daa40, 0xc208164000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080daa80, 0xc208164000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080e0480, 0xc2080da900, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e0480, 0x80f160, 0xc2080dab40, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 41 [IO wait]:
net.(*pollDesc).Wait(0xc2080f4a70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080f4a70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080f4a10, 0x0, 0x7f94f5639df0, 0xc2f904d1a8)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc2080f2088, 0xc208065e38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc2080f2088, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc2080e2ae0, 0x7f94f563d8a0, 0xc2080f2088, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 22 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc208082000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 23 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc208082000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 42 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc2080115d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080115d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011570, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc2080d1ca8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038178, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc208038178, 0xc208076898, 0xc2080c5000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801fec0, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080667e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080667e0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080667e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080d3080)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080d3080, 0xc2080e7200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080fe1c0, 0xc2080e7200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080fe1c0, 0xc2080e7200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080fe2c0, 0xc2080e7200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc208160000, 0xc2080da740, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc208160000, 0x80f160, 0xc2080fe380, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 1399 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0x12d, 0xc2d067e500, 0xec, 0x100, 0xc2d09b6000, 0xcf, 0x100, 0xc2d09f9980, 0xc2d09f98c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1050 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0xcc, 0xc24d6a4f00, 0xec, 0x100, 0xc255fdc000, 0xb7, 0x100, 0xc252e4c6c0, 0xc252e4c600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4453 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2667a3080, 0x3b9aca00, 0xc3ca29daf0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3ca011130, 0x7f94f5643c78, 0xc3ca005680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f94f5643c78, 0xc3ca005680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 53 [runnable]:
net.(*pollDesc).Wait(0xc2080f4fb0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080f4fb0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080f4f50, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc2769b4208)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038228, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc208038228, 0xc208076478, 0xc2080c3000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080fbb80, 0xc2080c3000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e3740)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e3740, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076420)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 54 [select]:
net/http.(*persistConn).writeLoop(0xc208076420)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 68 [IO wait]:
net.(*pollDesc).Wait(0xc2080f4990, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080f4990, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080f4930, 0xc2081d6000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc273521588)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038148, 0xc2081d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc208038148, 0xc2080761b8, 0xc2081d6000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081d4be0, 0xc2081d6000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e2b40)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e2b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076160)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 69 [select]:
net/http.(*persistConn).writeLoop(0xc208076160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 266 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xc9, 0xc20816d100, 0xec, 0x100, 0xc217b2c000, 0xab, 0x100, 0xc210b51440, 0xc210b51380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2624 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6c900, 0x1, 0xc3616db400, 0xe9, 0x100, 0xc361c7d200, 0x21, 0x40, 0xc361c83200, 0xc361c83140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1348 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0x12c, 0xc2ad8e3f00, 0xec, 0x100, 0xc2b4268000, 0xbc, 0x100, 0xc2b42cba40, 0xc2b42cb980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 660 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88fc0, 0xc8, 0xc262eca900, 0xec, 0x100, 0xc25754e000, 0xbb, 0x100, 0xc266e3fd40, 0xc266e3e300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 590 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88d80, 0x0, 0xc24cdba400, 0xea, 0x100, 0xc24c961800, 0xb8, 0x100, 0xc24ce43740, 0xc24ce43680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4270 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ca0ef440, 0x0, 0xc3d4276700, 0xea, 0x100, 0xc3d3d9b000, 0xa0, 0x100, 0xc3d42fd200, 0xc3d42fd140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 584 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88fc0, 0x12d, 0xc233717300, 0xec, 0x100, 0xc2489b7800, 0xcf, 0x100, 0xc249281860, 0xc2492817a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 974 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x1, 0xc291ca6100, 0xea, 0x100, 0xc2912b3000, 0xc7, 0x100, 0xc291ca9860, 0xc291ca97a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 849 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0x12c, 0xc273042500, 0xec, 0x100, 0xc273254c00, 0x24, 0x40, 0xc27311bb00, 0xc27311ba40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 951 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x65, 0xc2769b2d00, 0xec, 0x100, 0xc27964b000, 0xc6, 0x100, 0xc279be2780, 0xc279be26c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1395 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x2, 0xc2ca23a100, 0xea, 0x100, 0xc2ca258000, 0xcf, 0x100, 0xc2ca605680, 0xc2ca6055c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1691 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca000, 0x65, 0xc262eca200, 0xec, 0x100, 0xc284bf3800, 0xc6, 0x100, 0xc24028dd40, 0xc24028dc80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 636 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88d80, 0x2, 0xc262ecaa00, 0xea, 0x100, 0xc2625f3000, 0xcf, 0x100, 0xc263118600, 0xc263118540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1959 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0xc9, 0xc30d5f5b00, 0xec, 0x100, 0xc30f8c6800, 0xbd, 0x100, 0xc30ff042a0, 0xc30ff041e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4120 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3ce8816e0, 0xc3cecf39e0, 0xc3cecf3a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 981 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x64, 0xc284df2100, 0xec, 0x100, 0xc284d30000, 0xc9, 0x100, 0xc284ea8a80, 0xc284ea89c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 296 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12e, 0xc20816de00, 0xec, 0x100, 0xc21f65b800, 0xb3, 0x100, 0xc21ede3f20, 0xc21ede3e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1333 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x64, 0xc2a9983200, 0xec, 0x100, 0xc2abeaa000, 0xc9, 0x100, 0xc2ac162de0, 0xc2ac162d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 662 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88d80, 0x1, 0xc262ecbe00, 0xea, 0x100, 0xc24a06d800, 0xc7, 0x100, 0xc26199cb40, 0xc26199ca80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4441 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ca0ef440, 0x0, 0xc23b44ef00, 0xea, 0x100, 0xc27450f800, 0xc8, 0x100, 0xc273ce7620, 0xc273ce7560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3407 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3984d7200, 0x64, 0xc3a9b62800, 0xec, 0x100, 0xc3a99d5000, 0xc9, 0x100, 0xc3a9f6ec00, 0xc3a9f6eb40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 674 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88fc0, 0x130, 0xc24cdbb600, 0xec, 0x100, 0xc25d29b000, 0xbc, 0x100, 0xc21b03e420, 0xc21b03e360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1754 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca480, 0xc9, 0xc233716100, 0xec, 0x100, 0xc2faae8000, 0xbd, 0x100, 0xc2fab348a0, 0xc2fab347e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 697 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88d80, 0x64, 0xc230c8d800, 0xec, 0x100, 0xc24ca0d000, 0xc9, 0x100, 0xc23bc7e9c0, 0xc23bc7e900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4422 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2e2dd6090, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:575 +0x192
github.com/couchbase/go-couchbase.getStatsParallel(0xc28c413ba0, 0x1, 0x9cac80, 0x3, 0xc2d4a6d740)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 2483 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0xc9, 0xc23bd0fc00, 0xec, 0x100, 0xc2fda2e800, 0xbd, 0x100, 0xc2fde6f1a0, 0xc2fde6f0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 135 [select]:
net/http.(*persistConn).writeLoop(0xc2080764d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 134 [IO wait]:
net.(*pollDesc).Wait(0xc20c726530, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c726530, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c7264d0, 0xc20c6a3000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc2fa8b4800)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038040, 0xc20c6a3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc208038040, 0xc208076528, 0xc20c6a3000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c6ef580, 0xc20c6a3000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e3560)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e3560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080764d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4112 [sleep]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3ca0ef440, 0xc3d3ff0000, 0xc3a43a5b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:760 +0x287
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3ca0ef440, 0xc3d3cd5f80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3ca0ef440, 0xc3d3cd5f80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3ca0ef440, 0xc3ca24a150, 0xc3ca0ef5e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 130 [select]:
net/http.(*persistConn).writeLoop(0xc2080822c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 112 [select]:
net/http.(*persistConn).writeLoop(0xc208082420)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 111 [runnable]:
net.(*pollDesc).Wait(0xc20c694680, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c694680, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c694620, 0xc20c69d000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc2f8e7bc90)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f2190, 0xc20c69d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc2080f2190, 0xc208082478, 0xc20c69d000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c6a1ac0, 0xc20c69d000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20c69a840)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20c69a840, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208082420)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2224 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0xc8, 0xc2d067e600, 0xeb, 0x100, 0xc2d5057200, 0x23, 0x40, 0xc2d7c4bc80, 0xc2d7c4bbc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 129 [IO wait]:
net.(*pollDesc).Wait(0xc20c6944c0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c6944c0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c694460, 0xc20c6a7000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc27a7833a0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f21a0, 0xc20c6a7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc2080f21a0, 0xc208082318, 0xc20c6a7000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c6a1b40, 0xc20c6a7000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20c69a960)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20c69a960, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080822c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1741 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca000, 0x4, 0xc23bd0e600, 0xea, 0x100, 0xc2f9b2b800, 0xa2, 0x100, 0xc2f9e16420, 0xc2f9e16360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2713 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6cb40, 0xca, 0xc36a5b8700, 0xec, 0x100, 0xc36b5b2000, 0x9e, 0x100, 0xc36b60a8a0, 0xc36b60a7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4421 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2e2e91f20, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:575 +0x192
github.com/couchbase/go-couchbase.getStatsParallel(0xc28c413ba0, 0x0, 0x9cac80, 0x3, 0xc2d4a6d740)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 4471 [runnable]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3cc3946c0, 0x7fed003d, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3ca005380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3ca005380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc279971a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4134 [semacquire]:
sync.(*RWMutex).RLock(0xc3cc3946c0)
	/usr/local/go/src/sync/rwmutex.go:36 +0x5f
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3cc3946c0, 0xc3cee29140)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:424 +0x4a7
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 1382 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0xc9, 0xc2c83aee00, 0xec, 0x100, 0xc2c7fd9000, 0xbd, 0x100, 0xc2c8adf260, 0xc2c8adf1a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2058 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0x12c, 0xc34d936d00, 0xec, 0x100, 0xc34d694800, 0xc4, 0x100, 0xc34dae1f20, 0xc34dae1e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2034 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0xca, 0xc337002700, 0xec, 0x100, 0xc336e45800, 0xbc, 0x100, 0xc3372c7920, 0xc3372c7860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2458 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0x12e, 0xc25f505e00, 0xec, 0x100, 0xc2efa11000, 0xb3, 0x100, 0xc346dc8420, 0xc346dc8360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 145 [syscall, 3 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 1029 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x69, 0xc262ecbf00, 0xec, 0x100, 0xc2902a6800, 0xba, 0x100, 0xc22c35fe00, 0xc22c35fd40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 923 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0xc8, 0xc2779fe700, 0xec, 0x100, 0xc277ad0000, 0xbb, 0x100, 0xc277ab7980, 0xc277ab78c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 335 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12d, 0xc230c8d700, 0xec, 0x100, 0xc238e1b000, 0xcf, 0x100, 0xc239462780, 0xc2394626c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 251 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x64, 0xc2116f2a00, 0xec, 0x100, 0xc211212000, 0xc6, 0x100, 0xc211f99680, 0xc211f995c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3526 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3984d7440, 0xc8, 0xc3b0518800, 0xec, 0x100, 0xc3afb95000, 0xa4, 0x100, 0xc3b0910240, 0xc3b0910180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1732 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca000, 0x3, 0xc273077d00, 0xea, 0x100, 0xc2eed2d800, 0xb5, 0x100, 0xc2ef656540, 0xc2ef656480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2767 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6cb40, 0x12e, 0xc389df3200, 0xec, 0x100, 0xc38a00c000, 0xb3, 0x100, 0xc38a0517a0, 0xc38a0516e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 310 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12f, 0xc214b8e600, 0xec, 0x100, 0xc22e5b0000, 0xce, 0x100, 0xc226fb9ce0, 0xc226fb9c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 654 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88d80, 0x65, 0xc23bd0ee00, 0xec, 0x100, 0xc25aab3000, 0xea, 0x100, 0xc22ea09740, 0xc22ea09680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1034 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x2, 0xc24cdbb800, 0xea, 0x100, 0xc2758da800, 0xcf, 0x100, 0xc23283d980, 0xc23283d8c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 630 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88fc0, 0x12c, 0xc25f504600, 0xec, 0x100, 0xc25f6b8000, 0xb4, 0x100, 0xc25f691680, 0xc25f6915c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2011 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0x131, 0xc32adec700, 0xec, 0x100, 0xc32a2d7000, 0xbf, 0x100, 0xc32b133800, 0xc32b133740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 501 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88d80, 0x65, 0xc230c8c600, 0xec, 0x100, 0xc2304e7800, 0xb9, 0x100, 0xc2262c4060, 0xc22c9d1f80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 695 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23bb88fc0, 0x12f, 0xc230c8cc00, 0xec, 0x100, 0xc248a3a800, 0xce, 0x100, 0xc25aca23c0, 0xc25aca2300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1717 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca480, 0xc8, 0xc25a854a00, 0xec, 0x100, 0xc27816d000, 0xbb, 0x100, 0xc2d94be540, 0xc2d94be480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 328 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x65, 0xc2116f3f00, 0xec, 0x100, 0xc22a9b6800, 0xc6, 0x100, 0xc228dd6780, 0xc228dd66c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1011 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0xcb, 0xc2779fef00, 0xec, 0x100, 0xc27c7fa800, 0xc9, 0x100, 0xc27d038900, 0xc27d038840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 987 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0x131, 0xc284df3b00, 0xec, 0x100, 0xc28fcc3000, 0xbf, 0x100, 0xc2902be8a0, 0xc2902be7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1241 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0x12c, 0xc273077400, 0xec, 0x100, 0xc273bdf800, 0xaa, 0x100, 0xc256f2b500, 0xc256f2b440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 964 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0x12c, 0xc27ba6b400, 0xec, 0x100, 0xc2831ed800, 0xaf, 0x100, 0xc2838cfc80, 0xc2838cfbc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1405 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0xca, 0xc2d8606500, 0xec, 0x100, 0xc2d81be800, 0xbc, 0x100, 0xc2d8887da0, 0xc2d8887ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 338 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x2, 0xc23b44e300, 0xea, 0x100, 0xc23b4c4000, 0xcf, 0x100, 0xc23b4cde00, 0xc23b4cdd40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 270 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x0, 0xc214b8e500, 0xea, 0x100, 0xc215d47000, 0xb8, 0x100, 0xc214413aa0, 0xc2144139e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 288 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x1, 0xc2116f3600, 0xea, 0x100, 0xc226c5e000, 0xc7, 0x100, 0xc226d27380, 0xc226d272c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1661 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca480, 0xcc, 0xc2ad8e2b00, 0xec, 0x100, 0xc2b1226000, 0xb7, 0x100, 0xc2c18da960, 0xc2c18da8a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3990 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3b51f1b00, 0xc8, 0xc3c4c27300, 0xec, 0x100, 0xc3c4ab1800, 0xbb, 0x100, 0xc3c5036660, 0xc3c50365a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2450 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0xca, 0xc291ca7c00, 0xec, 0x100, 0xc2b18f8800, 0xbc, 0x100, 0xc2cafe3e00, 0xc2cafe3d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1211 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x0, 0xc20816cb00, 0xea, 0x100, 0xc273198c00, 0x3f, 0x40, 0xc298aec720, 0xc298aec660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2052 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ab40, 0x4, 0xc347f2d100, 0xea, 0x100, 0xc34cc60000, 0xa2, 0x100, 0xc34cbcfd40, 0xc34cbcfc80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1015 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x0, 0xc2769b2b00, 0xea, 0x100, 0xc2904b9000, 0x9e, 0x100, 0xc2906c4840, 0xc2906c4780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1783 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca000, 0x1, 0xc2fda18000, 0xea, 0x100, 0xc2fd958000, 0xc7, 0x100, 0xc2fd9b5980, 0xc2fd9b58c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4137 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3d34b4240, 0xc3d34ae5a0, 0xc3d34ae600)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 2082 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0x12d, 0xc34f26f200, 0xec, 0x100, 0xc34f2c4800, 0xcf, 0x100, 0xc34fc78d80, 0xc34fc78cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1341 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x0, 0xc2b7dfdc00, 0xea, 0x100, 0xc2b8e2f800, 0x9f, 0x100, 0xc2b93a5680, 0xc2b93a55c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2416 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x6a, 0xc291ca7a00, 0xec, 0x100, 0xc324771000, 0xcc, 0x100, 0xc31595a780, 0xc31595a6c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1414 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x65, 0xc2d0feaf00, 0xec, 0x100, 0xc2d2020000, 0xc6, 0x100, 0xc2d23b20c0, 0xc2d23b2000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2501 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x4, 0xc2116f3800, 0xea, 0x100, 0xc34c655800, 0xa2, 0x100, 0xc2fd3fe960, 0xc2fd3fe8a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2454 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x69, 0xc262eca700, 0xec, 0x100, 0xc3116db000, 0xba, 0x100, 0xc316e0c660, 0xc316e0c5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1985 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0xc8, 0xc31e84a100, 0xec, 0x100, 0xc31dcf5000, 0xbb, 0x100, 0xc31e9c52c0, 0xc31e9c5200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1062 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0xce, 0xc23b44fe00, 0xec, 0x100, 0xc2742ad000, 0xab, 0x100, 0xc25a93acc0, 0xc25a93ac00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 861 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x0, 0xc273a2b100, 0xea, 0x100, 0xc273983000, 0x64, 0x80, 0xc273b502a0, 0xc273b501e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2631 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6cb40, 0xca, 0xc361fac800, 0xec, 0x100, 0xc36206e000, 0x79, 0x80, 0xc3621ad260, 0xc3621ad1a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1768 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca000, 0x66, 0xc21f02d600, 0xec, 0x100, 0xc2fb62f000, 0xc8, 0x100, 0xc2fba381e0, 0xc2fba38120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2656 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6c900, 0x2, 0xc365c87900, 0xea, 0x100, 0xc36678c000, 0xcf, 0x100, 0xc36687ce40, 0xc36687cd80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1905 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0x12e, 0xc30831b300, 0xec, 0x100, 0xc3080d3200, 0x34, 0x40, 0xc308720720, 0xc308720660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1025 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9680, 0xcd, 0xc284df3900, 0xec, 0x100, 0xc290bb5800, 0xba, 0x100, 0xc2337615c0, 0xc233761500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1075 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2571a9440, 0x66, 0xc230c8cb00, 0xec, 0x100, 0xc2737f1000, 0xc8, 0x100, 0xc27f9652c0, 0xc27f965200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1738 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca480, 0xcd, 0xc24cdbbd00, 0xec, 0x100, 0xc2f4a79000, 0xba, 0x100, 0xc2f5428240, 0xc2f5428180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4426 [IO wait]:
net.(*pollDesc).Wait(0xc2c2a3fa30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2c2a3fa30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2c2a3f9d0, 0xc257225000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc212557670)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080389b8, 0xc257225000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f94f563bcf8, 0xc2080389b8, 0xc2f95ecf78, 0xc257225000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2d5918de0, 0xc257225000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2d97f7bc0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2d97f7bc0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2f95ecf20)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1698 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca000, 0x0, 0xc2908ccd00, 0xea, 0x100, 0xc2d5aba000, 0xb8, 0x100, 0xc2334ff320, 0xc2334ff260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2044 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ad80, 0x12f, 0xc346704900, 0xec, 0x100, 0xc34610d000, 0xce, 0x100, 0xc3467565a0, 0xc3467564e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1996 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ab40, 0x2, 0xc330d9c300, 0xea, 0x100, 0xc330a3f800, 0xcf, 0x100, 0xc330feac00, 0xc330feab40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2408 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0x12f, 0xc2b7dfc700, 0xec, 0x100, 0xc333e1f000, 0xce, 0x100, 0xc2f039fa40, 0xc2f039f980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1452 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0xcf, 0xc2de1c0c00, 0xec, 0x100, 0xc2dfed9000, 0xac, 0x100, 0xc2e04e9ce0, 0xc2e04e9c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1352 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456fc0, 0xc8, 0xc2b48d3c00, 0xec, 0x100, 0xc2bdf75800, 0xbb, 0x100, 0xc2be2e2960, 0xc2be2e28a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2005 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ab40, 0x66, 0xc31e84a900, 0xec, 0x100, 0xc32482c000, 0xc8, 0x100, 0xc32497f4a0, 0xc32497f3e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1903 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ab40, 0x0, 0xc30831b000, 0xea, 0x100, 0xc307c68800, 0xb8, 0x100, 0xc3085ddbc0, 0xc3085ddb00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4423 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2e2dd61b0, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:575 +0x192
github.com/couchbase/go-couchbase.getStatsParallel(0xc28c413ba0, 0x2, 0x9cac80, 0x3, 0xc2d4a6d740)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 2488 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x4, 0xc21f02dc00, 0xea, 0x100, 0xc2fca36800, 0xa2, 0x100, 0xc2f9976b40, 0xc2f9976a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1409 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x1, 0xc2d8607b00, 0xea, 0x100, 0xc2d91fb800, 0xc7, 0x100, 0xc2d9541560, 0xc2d95414a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1435 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x3, 0xc2df0d9e00, 0xea, 0x100, 0xc2e0bf2800, 0xb5, 0x100, 0xc2e1e40840, 0xc2e1e40780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4455 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2667a34e0, 0x3b9aca00, 0xc3ca29daf0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3ca011130, 0x7f94f5643c78, 0xc3ca005380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f94f5643c78, 0xc3ca005380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 1356 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25b456d80, 0x66, 0xc2be4e1d00, 0xec, 0x100, 0xc2c1870000, 0xc8, 0x100, 0xc2c1a78c00, 0xc2c1a78b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1775 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca480, 0x12f, 0xc20816c200, 0xec, 0x100, 0xc2fc8d8000, 0xce, 0x100, 0xc2fc9c2300, 0xc2fc9c2240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1704 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a3bca480, 0x12c, 0xc2779fe500, 0xec, 0x100, 0xc2ce862800, 0xb5, 0x100, 0xc27d18cde0, 0xc27d18cd20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2406 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x1, 0xc330d9c400, 0xea, 0x100, 0xc33723b000, 0xc7, 0x100, 0xc2ec497680, 0xc2ec4975c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2036 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ab40, 0x68, 0xc33e79aa00, 0xec, 0x100, 0xc33e533800, 0xc2, 0x100, 0xc33efd0720, 0xc33efd0660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2062 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe86ab40, 0x1, 0xc34e8db200, 0xea, 0x100, 0xc34e355000, 0xc7, 0x100, 0xc34ebc69c0, 0xc34ebc6900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2374 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x64, 0xc337003b00, 0xec, 0x100, 0xc2feb27800, 0xc8, 0x100, 0xc326d0e060, 0xc326dcdf80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2754 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6cb40, 0x12d, 0xc37e968500, 0xec, 0x100, 0xc37ea8c000, 0xcf, 0x100, 0xc37ebe27e0, 0xc37ebe2720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2780 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6c900, 0x65, 0xc390b6a500, 0xec, 0x100, 0xc391386800, 0xc6, 0x100, 0xc391bec240, 0xc391bec180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2765 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6c900, 0x1, 0xc382bc1700, 0xea, 0x100, 0xc386b39000, 0xb1, 0x100, 0xc386fd9980, 0xc386fd98c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2343 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0x12c, 0xc30a601100, 0xec, 0x100, 0xc3098cb000, 0xc4, 0x100, 0xc30c54f920, 0xc30c54f860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2499 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0x131, 0xc2116f2f00, 0xec, 0x100, 0xc2fbc37000, 0xbf, 0x100, 0xc2fac4f740, 0xc2fac4f680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2398 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0xcc, 0xc347f2c400, 0xec, 0x100, 0xc340521800, 0xb7, 0x100, 0xc2b53caa80, 0xc2b53ca9c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2732 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6cb40, 0x12c, 0xc373ca6400, 0xec, 0x100, 0xc3728f0800, 0xc4, 0x100, 0xc373d2f980, 0xc373d2f8c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4427 [select]:
net/http.(*persistConn).writeLoop(0xc2f95ecf20)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2388 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df680, 0xc8, 0xc34f26ef00, 0xec, 0x100, 0xc2b4b3d000, 0xa8, 0x100, 0xc31108bce0, 0xc31108bc20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2382 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x65, 0xc34e8dbf00, 0xec, 0x100, 0xc3414cc000, 0xc6, 0x100, 0xc248440ba0, 0xc25f180840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2463 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2122df440, 0x5, 0xc273a2b200, 0xea, 0x100, 0xc2fda2a000, 0xbc, 0x100, 0xc2fda6f1a0, 0xc2fda6f0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2728 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6c900, 0x64, 0xc37056c800, 0xec, 0x100, 0xc36f8c4800, 0xc9, 0x100, 0xc37094ca20, 0xc37094c960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2746 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6c900, 0x0, 0xc37d08a200, 0xea, 0x100, 0xc37cfe4000, 0xb8, 0x100, 0xc37d0b88a0, 0xc37d0b87e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3530 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3984d7200, 0x0, 0xc3b1034a00, 0xea, 0x100, 0xc3b1310000, 0xb8, 0x100, 0xc3b130dec0, 0xc3b130de00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2791 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc317c6cb40, 0xc8, 0xc391fca700, 0xec, 0x100, 0xc391e2b800, 0xbb, 0x100, 0xc392091260, 0xc3920911a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4113 [semacquire]:
sync.(*WaitGroup).Wait(0xc3ca0ef5e0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc3ca0ef440)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3ca0ef440)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4472 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3ca0ef440, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3ca0ef440, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc279971b00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4380 [chan receive]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc28c413ba0, 0x9cac80, 0x3, 0xc2080d1570)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc3ca1f8840, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc3ca1f8840, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3d6102c60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4401 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ca0ef680, 0x12e, 0xc233717600, 0xec, 0x100, 0xc27421d000, 0x9f, 0x100, 0xc2a36bd3e0, 0xc2a36bd2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4333 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ca0ef680, 0x12e, 0xc3d5f42500, 0xeb, 0x100, 0xc3d5fa0000, 0x1e, 0x20, 0xc3d5e610e0, 0xc3d5e61020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4451 [IO wait]:
net.(*pollDesc).Wait(0xc2f4591560, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2f4591560, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2f4591500, 0xc308b11000, 0x1000, 0x1000, 0x0, 0x7f94f5639df0, 0xc2fa977a20)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc23b8ed090, 0xc308b11000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc23bcf69a8, 0xc308b11000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc25e53ff20, 0xc308b11000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc3d6102180)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc3d6102180, 0xc24680c30a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc3d6102180, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc23a551d70, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc23a551d70, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc3d6102180, 0xc262fca4e0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc23bcf6960, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc23bcf6960)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3902 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3b51f1b00, 0x12c, 0xc3c1171d00, 0xec, 0x100, 0xc3c26f0800, 0xc4, 0x100, 0xc3c2f0d200, 0xc3c2f0d140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4456 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2667a36a0, 0x3b9aca00, 0xc3ca29daf0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3ca011130, 0x7f94f56437f8, 0xc3ca0ef440, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f94f56437f8, 0xc3ca0ef440)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 4469 [runnable]:
sync.(*RWMutex).Lock(0xc3ca553560)
	/usr/local/go/src/sync/rwmutex.go:87 +0xa4
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3ca553560, 0x7fed00d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:309 +0x56
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3ca005680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3ca005680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc2799718c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4436 [IO wait]:
net.(*pollDesc).Wait(0xc258cd60d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc258cd60d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc258cd6070, 0xc245017900, 0x12c, 0x12c, 0x0, 0x7f94f5639df0, 0xc2f8da26f8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc34f5cddc8, 0xc245017900, 0x12c, 0x12c, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
github.com/couchbase/goxdcr/parts.(*CapiNozzle).tcpProxy(0xc3ca0ef680, 0x12e, 0xc2f00e09c0, 0xc2f2c2a620, 0xc2f00e0900, 0xc2f00e0960)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:923 +0xa52
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:831 +0xc91

goroutine 3893 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3b51f1b00, 0x12d, 0xc3bffe4e00, 0xec, 0x100, 0xc3becfe800, 0xcf, 0x100, 0xc3c02e1b60, 0xc3c02e1aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4428 [select]:
net/http.(*persistConn).roundTrip(0xc208082420, 0xc2f8db0fc0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc26668e5b0, 0xc2667a2f20, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc26668e5b0, 0x7f94f5639fe8, 0xe94600, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc26668e5b0, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc26668e5b0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc26668e5b0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc26668e5b0, 0x7f94f563db20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc2bf12abd0, 0x9bbca0, 0x6, 0x7f94f563db20, 0xc2667a2de0, 0x80f820, 0xc2799717b8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc2799717a0, 0x9bbca0, 0x6, 0x80f820, 0xc2799717b8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc2667a2d60, 0x15, 0x7f94f563db20, 0xc2667a2de0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/goxdcr/utils.LocalPool(0xc2f8db0f70, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:85 +0x263
github.com/couchbase/goxdcr/utils.LocalBucket(0xc2f8db0f70, 0xe, 0xc2080d1570, 0x8, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:105 +0x166
github.com/couchbase/goxdcr/service_impl.(*ClusterInfoSvc).GetBucket(0xc2080380f0, 0x7f94f563e030, 0xc20801f440, 0xc2080d1570, 0x8, 0x40adf9, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/cluster_info_service.go:133 +0x25e
github.com/couchbase/goxdcr/factory.(*XDCRFactory).constructSourceNozzles(0xc2080325a0, 0xc3a5f1da00, 0xc20800b7c0, 0x32, 0xc2f8db0f50, 0x1, 0xc2f4e0db00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:280 +0x11c
github.com/couchbase/goxdcr/factory.(*XDCRFactory).NewPipeline(0xc2080325a0, 0xc20800b7c0, 0x32, 0xc2f8db0e60, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:94 +0x39b
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).startPipeline(0xeb0fa0, 0xc20800b7c0, 0x32, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:280 +0x413
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc2773ebf00, 0xc2d97f7f80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:593 +0xdc2
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc2773ebf00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 4171 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3ca005680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3598 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3984d7200, 0x0, 0xc3b4b96d00, 0xea, 0x100, 0xc3b45e9000, 0xb8, 0x100, 0xc3b4cd9380, 0xc3b4cd92c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4017 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3b51f1b00, 0x12e, 0xc3c9919200, 0xec, 0x100, 0xc3c9a8c000, 0xb3, 0x100, 0xc3c9aa9560, 0xc3c9aa94a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4000 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3b51f1b00, 0xc9, 0xc3c6a37b00, 0xec, 0x100, 0xc3c814a000, 0xbd, 0x100, 0xc3c7dc0d20, 0xc3c7dc0c60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4173 [semacquire]:
sync.(*RWMutex).RLock(0xc3ca553560)
	/usr/local/go/src/sync/rwmutex.go:36 +0x5f
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3ca553560, 0xc3d32bfe00)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:424 +0x4a7
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 4132 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3ca005380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75
[goport] 2017/04/25 13:03:07 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:03:07.632-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:03:07.632-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:03:07.632-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:03:07.662-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:03:07.665-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:03:07.665-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:03:07.665-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:03:07.665-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:03:07.665-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:03:07.665-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:03:07.665-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:03:07.665-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:03:07.665-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:03:07.665-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:03:07.665-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:03:07.665-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:03:07.665-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:03:07.665-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:03:07.665-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:03:07.665-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:03:07.665-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:03:07.666-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:03:07.666-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:03:07.666-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:03:07.668-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:03:07.668-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:03:07.668-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:03:07.669-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:03:08.082-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:03:22.712-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:03:22.717-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:03:22.719-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:03:22.826-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:03:22.826-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:03:22.826-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:03:22.870-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:03:22.870-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:03:22.870-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:03:22.870-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:03:22.870-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:03:22.870-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:03:22.871-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:03:22.873-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:03:22.878-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
GenericSupervisor 2017-04-25T13:03:22.880-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:03:22.880-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
GenericSupervisor 2017-04-25T13:03:22.880-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:22.880-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:22.880-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:22.880-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:22.880-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:03:22.880-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
CheckpointManager 2017-04-25T13:03:22.880-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
CapiUtils 2017-04-25T13:03:22.910-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:03:22.910-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:03:22.910-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:03:22.911-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
StatisticsManager 2017-04-25T13:03:22.911-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:03:22.911-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
XDCRFactory 2017-04-25T13:03:22.912-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:03:22.912-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476 is constructed, start it
GenericPipeline 2017-04-25T13:03:22.916-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[47 73 84 164 66 29 121 152 2 52 77 151 137 71 114 122 129 54 59 138 11 34 74 13 99 113 146 8 102 144 89 10 57 91 115 157 41 83 145 158 63 78 79 125 32 147 5 20 126 22 88 120 149 9 38 50 90 170 161 165 101 127 150 160 80 95 16 56 110 40 55 105 112 19 93 140 31 43 48 70 134 45 141 132 133 85 92 107 109 24 25 143 36 148 166 28 51 62 68 104 46 86 116 167 37 49 76 139 44 100 118 128 136 35 61 67 124 108 169 4 6 26 69 75 82 156 153 154 155 3 64 119 131 12 106 81 60 27 30 98 1 97 103 87 96 111 135 18 21 142 0 39 72 130 42 53 65 7 14 17 23 33 94 159 15 117 162 163 58 123 168] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[187 294 171 198 229 257 266 189 224 233 305 276 281 336 193 290 197 172 230 272 285 291 275 211 273 340 175 206 251 190 183 194 293 201 244 254 258 262 327 202 213 216 250 270 277 186 226 335 219 207 246 297 315 320 331 316 181 278 242 173 176 239 332 329 287 313 214 252 174 314 324 326 203 205 236 268 184 179 222 330 200 308 323 220 307 210 325 321 215 243 217 245 261 267 180 208 319 306 328 235 256 283 312 282 249 260 338 232 269 289 298 204 295 301 247 264 218 317 209 286 178 228 212 265 225 263 241 255 237 288 296 318 191 199 188 238 299 223 240 248 259 302 337 196 271 334 304 311 221 274 284 339 280 309 231 292 177 234 279 300 310 195 227 253 303 322 182 185 192 333]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[36 148 166 28 51 62 68 104 46 86 116 167 37 49 76 44 100 118 128 139 35 61 67 124 136 4 6 26 69 108 169 75 82 156 155 3 64 119 131 153 154 12 106 81 60 27 30 98 1 97 103 87 96 111 135 18 21 142 0 39 72 130 65 7 14 17 23 42 53 33 94 159 15 117 162 163 58 123 168 47 73 84 164 66 29 121 2 52 77 151 152 71 114 122 129 137 54 59 138 11 34 74 13 99 113 146 8 102 144 89 10 57 91 115 157 41 83 145 158 63 78 79 125 32 147 5 20 126 22 88 120 149 9 38 50 90 170 161 101 127 150 160 165 80 95 16 56 110 40 55 105 112 19 93 140 31 43 48 70 134 45 141 132 133 85 92 107 109 24 25 143] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[232 269 289 298 301 204 295 247 264 209 218 317 286 178 228 265 212 225 241 263 255 296 318 191 199 237 288 188 238 299 223 240 248 337 196 259 302 271 334 221 274 284 304 311 339 231 280 309 292 177 234 310 195 279 300 303 322 182 227 253 185 192 333 187 294 198 229 171 257 266 224 233 189 305 336 193 276 281 197 290 272 285 172 230 291 211 275 175 206 251 273 340 190 293 183 194 201 244 254 327 202 258 262 250 270 213 216 277 186 226 219 335 297 315 207 246 320 331 316 181 278 242 176 173 239 332 329 214 252 287 313 324 326 174 314 236 268 184 203 205 179 222 200 308 323 330 220 307 210 325 215 243 321 261 267 180 208 217 245 319 306 235 256 283 312 328 282 249 260 338]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression: active:true doc_batch_size_kb:2048 target_nozzle_per_node:2 replication_type:capi checkpoint_interval:1800 worker_batch_size:500]
CheckpointManager 2017-04-25T13:03:22.916-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476...
AsyncListener 2017-04-25T13:03:22.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:22.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:03:22.924-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:22.924-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:03:22.924-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:03:22.924-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:03:22.924-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:03:22.924-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:03:22.924-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:03:22.924-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:03:22.924-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:03:22.933-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:03:22.933-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:03:22.933-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:03:22.933-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:03:22.933-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:03:23.123-05:00 [INFO] validateSourceTopology...
TCPConnPoolMgr 2017-04-25T13:03:23.125-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:03:23.125-05:00 [INFO] connectionSize=5
CapiUtils 2017-04-25T13:03:23.127-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:03:23.127-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:03:23.127-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:03:23.128-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:03:23.128-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:03:23.128-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:03:23.128-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:03:23.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:03:23.143-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:03:23.143-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:03:23.143-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:03:23.201-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:03:23.202-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:03:23.202-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:03:23.202-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:03:23.202-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:03:23.202-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:03:23.203-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:03:23.307-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:03:23.307-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:03:23.309-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:03:23.331-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:03:23.331-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:03:23.484-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:03:23.484-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:03:23.484-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:03:23.490-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:03:23.490-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:03:23.491-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:03:23.491-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476 is started----------
PipelineManager 2017-04-25T13:03:23.491-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:03:23.491-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:03:23.491-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:03:24.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:03:24.149-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:03:24.149-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:03:24.150-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:03:24.150-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:03:24 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 121 successful
CapiNozzle 2017-04-25T13:03:24.475-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:03:24.477-05:00 [INFO] Send batch count=76
2017/04/25 13:03:24 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 27 successful
CapiNozzle 2017-04-25T13:03:24.669-05:00 [INFO] Send batch count=1
2017/04/25 13:03:24 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 149 successful
CapiNozzle 2017-04-25T13:03:24.846-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:24.875-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:24.902-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:03:24 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:03:24 UPR_STREAMREQ for vb 295 successful
CheckpointManager 2017-04-25T13:03:24.947-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
CheckpointManager 2017-04-25T13:03:24.947-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:03:25 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:03:25 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:03:25 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:03:25 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:03:25 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:03:25 UPR_STREAMREQ for vb 297 successful
CapiNozzle 2017-04-25T13:03:25.035-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
StatisticsManager 2017-04-25T13:03:25.058-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=5, changes_left=315503
CapiNozzle 2017-04-25T13:03:25.064-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
DcpNozzle 2017-04-25T13:03:25.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:03:25.103-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:03:25.279-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:25.289-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:03:25.375-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:25.388-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:03:25.711-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:25.711-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:03:25.711-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:03:25.711-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]
ReplicationManager 2017-04-25T13:03:25.711-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]
CapiNozzle 2017-04-25T13:03:25.714-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:25.734-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
PipelineManager 2017-04-25T13:03:25.776-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:03:25.777-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:03:25.781-05:00 [INFO] err_list=[{"time":"2017-04-25T13:03:25.781878205-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]
PipelineManager 2017-04-25T13:03:25.782-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:03:25.847-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:25.848-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:25.858-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
StatisticsManager 2017-04-25T13:03:25.943-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:03:25.944-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476 {"Errors": "[{\"time\":\"2017-04-25T13:03:25.781878205-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315503, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 30, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 344, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 5, "docs_received_from_dcp": 7423, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 344, "set_filtered": 0, "set_received_from_dcp": 7423, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:03:26.225-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:26.236-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:03:26.514-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:26.525-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:03:26.715-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:26.715-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:03:26.715-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:03:26.715-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:26.715-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:26.811-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:26.811-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:26.822-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:03:26.940-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:26.952-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:03:27.168-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:27.200-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:03:27.301-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:27.301-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:03:27.301-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:03:27.301-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:27.301-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:27.477-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:27.478-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:27.579-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:03:27.627-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:27.657-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:03:27.990-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:03:28.004-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:03:28.144-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:28.144-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:03:28.144-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:03:28.144-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:28.144-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:28.235-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:28.235-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:28.258-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:03:28.381-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:03:28.406-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:03:28.608-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:28.621-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:03:29.137-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:03:29.137-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:03:29.137-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:03:29.137-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:29.137-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:29.218-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:29.218-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:29.251-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:03:29.375-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:29.397-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:03:29.597-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:03:29.622-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:03:29.926-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:29.926-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:03:29.926-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:03:29.926-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:29.926-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:30.053-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:30.053-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:30.071-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:03:30.091-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:03:30.107-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:03:30.507-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:03:30.524-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:03:30.732-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:03:30.732-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:03:30.732-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:03:30.732-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:30.732-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:30.888-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:30.889-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:30.910-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:03:31.035-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:03:31.054-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:03:31.380-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:31.402-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:03:31.768-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:03:31.768-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:03:31.768-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:03:31.768-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:31.768-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:31.837-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:31.837-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:31.863-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:03:31.989-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:32.013-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:03:32.195-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:03:32.220-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:03:32.429-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:32.429-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:03:32.429-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:03:32.429-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:32.429-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:32.560-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:32.560-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:32.584-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:03:32.610-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:03:32.633-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
GenericSupervisor 2017-04-25T13:03:32.932-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:03:32.932-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:03:32.932-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:03:33.018-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:03:33.041-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
ToplogyChangeDetector 2017-04-25T13:03:33.212-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:03:33.314-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:03:33.314-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:03:33.314-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:03:33.314-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:33.314-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:33.429-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:33.429-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:33.455-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:03:33.469-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:03:33.506-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:03:33.894-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:03:33.911-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:03:34.091-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:03:34.091-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:03:34.091-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:03:34.091-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:34.091-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:34.183-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:34.183-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:34.203-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:03:34.259-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:03:34.283-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:03:34.561-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:34.578-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:03:34.770-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:03:34.770-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:03:34.770-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:03:34.770-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:34.770-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:34.873-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:34.873-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:34.889-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:03:35.135-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:35.156-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:03:35.474-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:03:35.485-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:03:35.674-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:35.674-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:03:35.674-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:03:35.674-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:35.674-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:35.757-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:35.757-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:35.768-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
PipelineManager 2017-04-25T13:03:35.790-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries 
PipelineManager 2017-04-25T13:03:35.790-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:03:35.790-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:03:35.790-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
PipelineRuntimeCtx 2017-04-25T13:03:35.790-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:03:35.790-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:35.790-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:03:35.790-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:35.79068502 -0500 CDT
GenericSupervisor 2017-04-25T13:03:35.790-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:35.790-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:03:35.790-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:03:35.791-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:03:35.791-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:03:35.791-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:03:35.791-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:03:35.791-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:03:35.791-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:03:35.791-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-871031476 has been stopped
GenericPipeline 2017-04-25T13:03:35.791-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-871031476 is stopped
PipelineManager 2017-04-25T13:03:35.791-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:03:35.791-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:03:25.781878205-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:03:35.791-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:03:35.792-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
GenericPipeline 2017-04-25T13:03:35.792-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
GenericPipeline 2017-04-25T13:03:35.792-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
GenericPipeline 2017-04-25T13:03:35.792-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-871031476
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:35.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:35.793-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:35.793-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:35.793-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:35.793-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:03:35.793-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:35.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:03:35.793-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:03:35.793-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:03:35.793-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:03:35.793-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:35.794-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:03:35.794-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:03:35.794-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:03:35.794-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:35.794-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:35.794-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:03:35.794-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:35.79400671 -0500 CDT
DcpNozzle 2017-04-25T13:03:35.794-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:03:35.794-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:35.794505819 -0500 CDT
CapiNozzle 2017-04-25T13:03:35.795-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:03:35.795-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:03:35.795-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:03:35.796-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:35.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:03:35.796-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:35.796-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:03:35.799-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:03:35.799-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:03:35.799-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:35.799599735 -0500 CDT
DcpNozzle 2017-04-25T13:03:35.799-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:35.799617138 -0500 CDT
DcpNozzle 2017-04-25T13:03:35.799-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:03:35.799-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:03:35.799-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:35.799-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:03:35.799-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:35.799-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:03:35 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:03:35 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:03:35.834-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:03:35.835-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:03:35.835-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:35.835-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:03:35.934-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:03:35.935-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:03:35.935-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:03:35.984-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:03:35.985-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:03:35.985-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:03:35.985-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:03:35.985-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:03:35.985-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:03:35.985-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:03:35.986-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:03:35.986-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
GenericSupervisor 2017-04-25T13:03:35.988-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:03:35.988-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
GenericSupervisor 2017-04-25T13:03:35.988-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:35.988-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:35.988-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:35.988-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:35.988-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:03:35.988-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
CheckpointManager 2017-04-25T13:03:35.988-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
CapiUtils 2017-04-25T13:03:36.019-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:03:36.019-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:03:36.019-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:03:36.020-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
StatisticsManager 2017-04-25T13:03:36.020-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:03:36.020-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
XDCRFactory 2017-04-25T13:03:36.020-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:03:36.021-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165 is constructed, start it
GenericPipeline 2017-04-25T13:03:36.022-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[279 316 190 291 336 198 242 193 339 212 235 296 333 200 298 218 278 206 273 276 189 216 287 289 329 309 337 208 223 252 290 322 181 195 301 268 294 199 334 191 238 308 251 280 209 220 240 282 303 335 176 219 178 272 203 234 263 277 267 274 245 254 299 317 332 249 271 310 183 312 318 323 201 222 228 231 292 338 302 175 194 246 259 221 232 182 265 186 284 184 233 177 192 258 260 293 304 331 213 224 225 247 262 306 275 288 230 204 244 250 311 226 261 180 202 205 328 187 196 227 256 283 305 295 179 255 269 321 326 243 281 285 171 210 324 327 239 297 319 174 229 266 248 315 173 313 214 264 320 215 286 185 307 207 241 172 188 325 340 197 217 237 314 211 330 270 257 236 300 253] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[68 102 106 157 45 169 141 88 66 121 52 85 64 166 76 58 119 30 75 124 160 3 27 118 7 63 137 140 147 65 108 99 111 112 125 146 159 37 41 126 142 148 158 20 32 90 129 19 70 24 40 152 153 155 17 77 92 117 67 135 143 4 12 110 156 29 104 53 39 51 36 130 170 49 56 13 16 86 87 107 5 9 91 59 11 28 120 138 103 122 134 57 95 79 127 136 164 43 114 25 94 10 14 71 81 132 144 80 131 62 73 1 128 0 98 21 100 105 47 69 38 61 60 123 149 167 23 46 154 163 54 96 101 18 44 83 97 165 48 55 82 89 2 34 33 109 115 168 15 31 74 145 139 26 93 22 42 150 162 6 113 35 161 50 133 78 116 8 72 84 151]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[88 66 52 85 121 64 166 76 58 119 75 124 160 3 27 30 7 63 118 140 147 65 108 137 111 112 125 146 159 37 41 99 126 148 158 20 32 142 129 19 70 90 24 40 153 155 17 77 152 92 117 135 143 4 12 67 156 29 104 110 39 51 53 36 130 49 56 170 13 16 87 107 5 9 86 91 11 28 59 120 138 122 134 57 95 103 79 127 164 43 114 136 94 10 14 25 71 81 144 80 131 132 62 73 1 0 98 128 21 100 47 69 105 38 61 123 149 167 23 46 60 163 54 96 154 18 44 101 97 165 48 55 83 89 2 34 82 109 115 168 15 31 33 74 145 26 93 139 22 42 162 6 113 150 35 161 50 133 116 8 72 78 84 151 68 102 106 157 45 169 141] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[218 298 278 206 273 276 216 287 289 329 189 337 208 309 252 290 223 322 181 195 268 294 301 199 334 191 238 308 280 209 220 251 240 282 303 335 176 219 272 178 203 234 263 267 274 277 254 299 317 332 245 249 271 310 183 312 318 323 201 222 228 231 292 338 302 175 194 246 259 232 221 182 265 284 184 186 177 192 233 258 260 293 304 331 224 213 225 247 262 275 288 306 230 204 244 250 311 226 261 180 202 205 328 196 227 256 283 305 187 295 255 269 321 326 179 281 285 171 243 210 324 327 297 319 239 174 229 266 315 248 173 313 214 264 320 215 286 185 307 207 241 188 325 340 172 197 237 314 217 211 330 257 270 236 300 253 316 279 190 336 198 242 291 193 339 235 296 333 200 212]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 active:true doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:03:36.022-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165...
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:03:36.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
CheckpointManager 2017-04-25T13:03:36.031-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:03:36.031-05:00 [INFO] Checkpointing starts in 1804.124 sec
PipelineRuntimeCtx 2017-04-25T13:03:36.031-05:00 [INFO] Service CheckpointManager has been started
CheckpointManager 2017-04-25T13:03:36.031-05:00 [INFO] checkpointing rountine started
StatisticsManager 2017-04-25T13:03:36.031-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:03:36.033-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:03:36.033-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:03:36.033-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:03:36.033-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:03:36.033-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:36.033-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:03:36.033-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:03:36.033-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:03:36.033-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:03:36.071-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:03:36.071-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:03:36.071-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:03:36.086-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:03:36.086-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:03:36.090-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:03:36.090-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:03:36.090-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:03:36.094-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:03:36.109-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:03:36.119-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:03:36.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:03:36.128-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:03:36.128-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:03:36.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:03:36.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:03:36.129-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:03:36.177-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:03:36.177-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:03:36.178-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:03:36.189-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:03:36.189-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:03:36.289-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:03:36.289-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:03:36.289-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:03:36.289-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:03:36.289-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:03:36.289-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:03:36.290-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165 is started----------
PipelineManager 2017-04-25T13:03:36.290-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:03:36.290-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:03:36.290-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:03:36.312-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:03:36.487-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:03:36.487-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:03:36.488-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:03:36.488-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:03:36 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 2 successful
CapiNozzle 2017-04-25T13:03:36.570-05:00 [INFO] Send batch count=59
CapiNozzle 2017-04-25T13:03:36.585-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:36.585-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:36.597-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:03:36 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 305 successful
CapiNozzle 2017-04-25T13:03:36.630-05:00 [INFO] Send batch count=1
2017/04/25 13:03:36 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:03:36 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:03:36.651-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:03:37.102-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
CapiNozzle 2017-04-25T13:03:37.109-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:37.146-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
2017/04/25 13:03:37 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 315 successful
CapiNozzle 2017-04-25T13:03:37.153-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:03:37.176-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
CapiNozzle 2017-04-25T13:03:37.187-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
StatisticsManager 2017-04-25T13:03:37.189-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:03:37 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 111 successful
CapiNozzle 2017-04-25T13:03:37.250-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
CapiNozzle 2017-04-25T13:03:37.278-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:03:37.278-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
CapiNozzle 2017-04-25T13:03:37.278-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:03:37.278-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]
ReplicationManager 2017-04-25T13:03:37.278-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]
PipelineManager 2017-04-25T13:03:37.297-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:03:37.297-05:00 [INFO] Send batch count=330
PipelineManager 2017-04-25T13:03:37.301-05:00 [INFO] err_list=[{"time":"2017-04-25T13:03:37.301641135-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries"}]
PipelineManager 2017-04-25T13:03:37.301-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:03:37.306-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:03:37 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 323 successful
CapiNozzle 2017-04-25T13:03:37.341-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:37.342-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:37.404-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:03:37.411-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:03:37 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 38 successful
CapiNozzle 2017-04-25T13:03:37.598-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:37.598-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:03:37.598-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:03:37.598-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:37.598-05:00 [INFO] Send batch count=500
2017/04/25 13:03:37 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 45 successful
CapiNozzle 2017-04-25T13:03:37.793-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:37.796-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:37.812-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
2017/04/25 13:03:37 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 239 successful
PipelineManager 2017-04-25T13:03:37.836-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:03:37.301641135-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]}
]
2017/04/25 13:03:37 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 244 successful
CapiNozzle 2017-04-25T13:03:37.941-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:03:37 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:03:37 UPR_STREAMREQ for vb 55 successful
CapiNozzle 2017-04-25T13:03:37.956-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
2017/04/25 13:03:38 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 252 successful
StatisticsManager 2017-04-25T13:03:38.040-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:03:38.040-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165 {"Errors": "[{\"time\":\"2017-04-25T13:03:37.301641135-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 21, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 3567, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 21, "set_filtered": 0, "set_received_from_dcp": 3567, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]", "Status": "Pending"}

2017/04/25 13:03:38 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 164 successful
CapiNozzle 2017-04-25T13:03:38.096-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
2017/04/25 13:03:38 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:03:38.113-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
2017/04/25 13:03:38 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 92 successful
CapiNozzle 2017-04-25T13:03:38.266-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:38.287-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:03:38 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 98 successful
CheckpointManager 2017-04-25T13:03:38.398-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
CheckpointManager 2017-04-25T13:03:38.398-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:03:38 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:03:38 UPR_STREAMREQ for vb 290 successful
DcpNozzle 2017-04-25T13:03:38.443-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:03:38 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:03:38.539-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:03:38.550-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
DcpNozzle 2017-04-25T13:03:38.591-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:03:38.678-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:38.678-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:03:38.678-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:03:38.679-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:38.679-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:38.818-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:38.818-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:38.835-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:03:39.182-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:03:39.182-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:03:39.182-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:03:39.182-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:39.182-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:39.244-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:39.249-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:39.275-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:03:39.518-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:03:39.561-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:03:40.119-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:40.152-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:03:40.334-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:03:40.348-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:03:41.239-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:41.251-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:03:41.411-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:03:41.411-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:03:41.411-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:03:41.411-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:41.411-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:41.495-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:41.495-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:41.517-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
CapiNozzle 2017-04-25T13:03:41.981-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:41.981-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:03:41.981-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:03:41.981-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:41.981-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:42.051-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:42.051-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:42.074-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:03:42.168-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:03:42.184-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
CapiNozzle 2017-04-25T13:03:42.419-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:42.439-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:03:42.623-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:03:42.659-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
CapiNozzle 2017-04-25T13:03:42.805-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:42.830-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:03:42.978-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:03:42.978-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
CapiNozzle 2017-04-25T13:03:42.978-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:03:42.978-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:42.978-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:43.068-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:43.068-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:43.085-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:03:43.964-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:43.978-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:43.978-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:03:43.978-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:03:43.978-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:43.978-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:43.991-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:03:44.105-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:44.105-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:44.144-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:03:44.401-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:44.429-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:03:44.546-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:03:44.567-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:03:44.768-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:44.768-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:03:44.768-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:03:44.768-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:44.768-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:44.840-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:44.840-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:44.864-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:03:45.005-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:03:45.019-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:03:45.285-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:45.309-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:03:45.594-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:03:45.594-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:03:45.594-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:03:45.594-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:45.595-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:45.650-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:45.650-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:45.660-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:03:45.878-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:45.889-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
GenericSupervisor 2017-04-25T13:03:46.033-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:03:46.033-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:03:46.034-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:03:46.094-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:46.105-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:03:46.213-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:46.213-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:03:46.213-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:03:46.213-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:46.213-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:03:46.290-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:03:46.313-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:46.313-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:46.324-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:03:46.324-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:46.335-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:03:46.574-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:03:46.574-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:03:46.574-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:03:46.574-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:46.574-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:46.633-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:46.634-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:46.644-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:03:46.761-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:46.772-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:03:46.957-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:03:46.968-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:03:47.172-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:47.183-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
PipelineManager 2017-04-25T13:03:47.301-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries 
PipelineManager 2017-04-25T13:03:47.302-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:03:47.302-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:03:47.302-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
PipelineRuntimeCtx 2017-04-25T13:03:47.302-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:03:47.302-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:47.302-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:03:47.302-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:47.302440885 -0500 CDT
GenericSupervisor 2017-04-25T13:03:47.302-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:47.302-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:03:47.302-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:03:47.302-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:03:47.302-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:03:47.303-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:03:47.303-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:03:47.303-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:03:47.303-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:03:47.303-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-985790165 has been stopped
GenericPipeline 2017-04-25T13:03:47.303-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-985790165 is stopped
PipelineManager 2017-04-25T13:03:47.303-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:03:47.303-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:03:37.301641135-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:03:47.304-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:03:47.304-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
GenericPipeline 2017-04-25T13:03:47.304-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:03:47.304-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:03:47.304-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-985790165
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:47.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:03:47.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:47.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:03:47.305-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:47.305-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:03:47.305-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:03:47.305-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:03:47.305-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:03:47.305-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:03:47.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:03:47.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:03:47.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:03:47.305-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:47.305715766 -0500 CDT
CapiNozzle 2017-04-25T13:03:47.305-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:47.305692133 -0500 CDT
CapiNozzle 2017-04-25T13:03:47.306-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:03:47.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:03:47.306-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:03:47.306-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:03:47.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:47.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:03:47.307-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:47.307-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:03:47.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:03:47.307-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:47.307-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:03:47.308-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:03:47.308-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:47.308798533 -0500 CDT
DcpNozzle 2017-04-25T13:03:47.308-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:03:47.308-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:47.308-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:03:47.310-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:03:47.310-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:03:47.310666956 -0500 CDT
DcpNozzle 2017-04-25T13:03:47.310-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:03:47.310-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:03:47.310-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:03:47 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:03:47 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:03:47 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:03:47 Stream Closed for vb 1 StreamEnd simulated
XDCRFactory 2017-04-25T13:03:47.419-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:03:47.420-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:03:47.420-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:03:47.518-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:03:47.518-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:03:47.518-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:03:47.518-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:03:47.518-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:03:47.518-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:03:47.518-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:03:47.518-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:03:47.519-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
GenericSupervisor 2017-04-25T13:03:47.520-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:03:47.520-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
GenericSupervisor 2017-04-25T13:03:47.520-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:47.520-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:47.520-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:47.520-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:03:47.520-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:03:47.520-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
CheckpointManager 2017-04-25T13:03:47.520-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
CapiUtils 2017-04-25T13:03:47.555-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:03:47.555-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:03:47.555-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:03:47.556-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
StatisticsManager 2017-04-25T13:03:47.556-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:03:47.556-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
XDCRFactory 2017-04-25T13:03:47.556-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:03:47.557-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274 is constructed, start it
GenericPipeline 2017-04-25T13:03:47.558-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[107 11 124 158 10 102 36 65 125 159 14 59 85 113 166 30 95 116 135 136 150 25 19 72 115 2 123 132 156 54 134 130 161 8 140 61 69 100 12 46 3 79 106 58 92 133 141 143 67 40 93 114 144 1 157 53 60 89 108 31 119 154 122 32 24 27 90 21 70 117 145 148 64 74 127 151 52 44 126 33 167 13 50 101 23 29 35 28 66 99 62 110 76 163 86 22 38 51 87 5 34 160 169 17 56 88 131 20 128 83 45 152 42 120 153 164 73 37 91 26 112 138 111 103 104 162 68 77 139 57 71 18 121 129 97 48 146 4 168 170 63 84 165 16 49 78 9 118 155 7 41 81 39 47 82 137 43 80 98 149 6 142 96 105 147 0 75 94 109 55 15] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[285 325 327 340 231 204 262 265 273 174 216 264 305 323 177 200 221 270 208 300 241 246 287 185 191 201 313 186 237 252 299 314 199 205 184 302 255 259 301 315 284 228 235 254 272 229 289 178 212 328 210 268 288 308 213 230 297 309 322 242 279 331 171 217 282 197 291 179 222 223 247 251 290 203 269 321 189 194 215 260 295 218 236 239 277 310 337 275 172 173 278 292 333 180 234 240 244 267 303 307 334 207 226 293 225 250 330 263 276 196 192 245 266 281 181 238 227 253 257 306 243 283 304 336 187 188 211 206 224 248 311 233 320 182 198 316 183 209 258 271 329 261 338 176 280 286 298 319 326 256 274 296 317 190 193 220 332 232 312 219 214 339 318 294 324 335 195 175 202 249]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[154 122 32 24 27 90 21 70 117 145 148 64 74 127 151 52 44 126 33 167 13 50 101 23 29 35 28 66 99 62 110 76 163 86 22 38 51 87 5 34 160 169 17 56 88 131 20 128 83 45 152 42 120 153 164 73 37 91 26 112 138 111 103 104 162 68 77 139 57 71 18 121 129 97 48 146 4 168 170 63 84 165 16 49 78 9 118 155 7 41 81 39 47 82 137 43 80 98 149 6 142 96 105 147 0 75 94 109 55 15 107 11 124 158 10 102 36 65 125 159 14 59 85 113 166 30 95 116 135 136 150 25 19 72 115 2 123 132 156 54 134 130 161 8 140 61 69 100 12 46 3 79 106 58 92 133 141 143 67 40 93 114 144 1 157 53 60 89 108 31 119] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[213 230 297 309 322 242 279 331 171 217 282 197 291 179 222 223 247 251 290 203 269 321 189 194 215 260 295 218 236 239 277 310 337 275 172 173 278 292 333 180 234 240 244 267 303 307 334 207 226 293 225 250 330 263 276 196 192 245 266 281 181 238 227 253 257 306 243 283 304 336 187 188 211 206 224 248 311 233 320 182 198 316 183 209 258 271 329 261 338 176 280 286 298 319 326 256 274 296 317 190 193 220 332 232 312 219 214 339 318 294 324 335 195 175 202 249 285 325 327 340 231 204 262 265 273 174 216 264 305 323 177 200 221 270 208 300 241 246 287 185 191 201 313 186 237 252 299 314 199 205 184 302 255 259 301 315 284 228 235 254 272 229 289 178 212 328 210 268 288 308]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info stats_interval:1000 replication_type:capi filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 optimistic_replication_threshold:256 checkpoint_interval:1800 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:03:47.558-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274...
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:03:47.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
StatisticsManager 2017-04-25T13:03:47.567-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:03:47.569-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:03:47.569-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:03:47.569-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:03:47.569-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:03:47.569-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:03:47.569-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:03:47.569-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:03:47.569-05:00 [INFO] Checkpointing starts in 1800.725 sec
PipelineRuntimeCtx 2017-04-25T13:03:47.569-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:03:47.569-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:03:47.570-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:03:47.570-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:03:47.573-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:03:47.573-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:03:47.573-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:03:47.573-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:03:47.575-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:03:47.578-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:03:47.620-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:03:47.620-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:03:47.620-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:03:47.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:03:47.640-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:03:47.640-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:03:47.646-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:03:47.681-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:03:47.681-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:03:47.681-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:03:47.681-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:03:47.681-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:03:47.681-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:03:47.732-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:03:47.732-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:03:47.749-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:03:47.771-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:03:47.771-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:03:47.915-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:03:47.915-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:03:47.915-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:03:47.938-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:03:47.938-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:03:47.938-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:03:47.938-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274 is started----------
PipelineManager 2017-04-25T13:03:47.938-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:03:47.938-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:03:47.938-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:03:48.741-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:03:49.218-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:03:49.219-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:03:49.219-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:03:49.219-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:03:49 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:03:49.353-05:00 [INFO] Send batch count=43
2017/04/25 13:03:49 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:03:49.383-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:49.383-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:49.396-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
2017/04/25 13:03:49 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 108 successful
CapiNozzle 2017-04-25T13:03:49.428-05:00 [INFO] Send batch count=140
CapiNozzle 2017-04-25T13:03:49.470-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:49.470-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:03:49 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 307 successful
CapiNozzle 2017-04-25T13:03:49.507-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:03:49.518-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
2017/04/25 13:03:49 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 117 successful
CapiNozzle 2017-04-25T13:03:49.549-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
2017/04/25 13:03:49 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 26 successful
StatisticsManager 2017-04-25T13:03:49.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:03:49.655-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
2017/04/25 13:03:49 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 321 successful
CapiNozzle 2017-04-25T13:03:49.678-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
2017/04/25 13:03:49 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 29 successful
CapiNozzle 2017-04-25T13:03:49.777-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
2017/04/25 13:03:49 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 238 successful
CapiNozzle 2017-04-25T13:03:49.818-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
2017/04/25 13:03:49 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 43 successful
CapiNozzle 2017-04-25T13:03:49.871-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:49.871-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:03:49.872-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:03:49.872-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]
ReplicationManager 2017-04-25T13:03:49.872-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]
2017/04/25 13:03:49 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:03:49 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 151 successful
CapiNozzle 2017-04-25T13:03:50.151-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:50.245-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
2017/04/25 13:03:50 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 169 successful
PipelineManager 2017-04-25T13:03:50.303-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:03:50.303-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:03:50.306-05:00 [INFO] err_list=[{"time":"2017-04-25T13:03:50.306168369-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries"}]
PipelineManager 2017-04-25T13:03:50.306-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:03:50 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 76 successful
CapiNozzle 2017-04-25T13:03:50.457-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:50.457-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:03:50 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 177 successful
CapiNozzle 2017-04-25T13:03:50.479-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:03:50.518-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:50.518-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:03:50.518-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:03:50.518-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:50.518-05:00 [INFO] Send batch count=500
2017/04/25 13:03:50 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 280 successful
StatisticsManager 2017-04-25T13:03:50.589-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:03:50.589-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274 {"Errors": "[{\"time\":\"2017-04-25T13:03:50.306168369-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 31, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 71, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 4863, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 71, "set_filtered": 0, "set_received_from_dcp": 4863, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:03:50.601-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:50.601-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:03:50 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 85 successful
CapiNozzle 2017-04-25T13:03:50.638-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
2017/04/25 13:03:50 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:03:50 UPR_STREAMREQ for vb 195 successful
DcpNozzle 2017-04-25T13:03:50.787-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CheckpointManager 2017-04-25T13:03:50.789-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
CheckpointManager 2017-04-25T13:03:50.789-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:03:50 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:03:50.941-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:03:50.991-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:03:51.024-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:03:51.125-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:51.194-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:03:51.452-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:03:51.473-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:03:51.582-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:51.606-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:03:51.858-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:03:51.858-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:03:51.858-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:03:51.858-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:51.858-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:51.975-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:51.975-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:51.995-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:03:52.053-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:03:52.053-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:03:52.053-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:03:52.053-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:52.053-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:52.277-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:52.277-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:52.322-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:03:52.422-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:52.456-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:03:52.856-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:03:52.875-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:03:53.140-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:53.180-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:03:53.249-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:03:53.300-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:03:53.680-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:03:53.680-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:03:53.680-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:03:53.680-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:53.680-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:03:53.819-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:03:50.306168369-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]}
]
CapiNozzle 2017-04-25T13:03:53.830-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:53.835-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:53.849-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:03:54.141-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:03:54.141-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:03:54.141-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:03:54.141-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:54.141-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:54.303-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:54.303-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:54.332-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:03:54.378-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:03:54.393-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:03:54.997-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:55.016-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:03:55.283-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:03:55.305-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:03:55.477-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:55.505-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:03:55.623-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:03:55.623-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:03:55.623-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:03:55.624-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:55.624-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:55.708-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:55.708-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:55.725-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:03:55.866-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:03:55.866-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:03:55.866-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:03:55.866-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:55.866-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:55.937-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:55.937-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:55.961-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:03:56.043-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:56.062-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:03:56.356-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:03:56.371-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:03:56.532-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:56.546-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:03:56.742-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:03:56.777-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:03:56.946-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:03:56.946-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:03:56.946-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:03:56.946-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:56.946-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:57.115-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:57.115-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:57.131-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:03:57.153-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:03:57.153-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:03:57.153-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:03:57.153-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:57.154-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:57.234-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:57.234-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:58.133-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:03:58.139-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
ToplogyChangeDetector 2017-04-25T13:03:58.147-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:03:58.155-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:03:58.155-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:03:58.159-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:03:58.217-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:03:58.682-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:03:58.705-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:03:58.942-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:58.974-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:03:59.103-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:03:59.103-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:03:59.103-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:03:59.103-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:59.103-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:59.157-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:59.157-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:59.170-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:03:59.430-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:59.452-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:03:59.637-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:03:59.656-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:03:59.849-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:03:59.849-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:03:59.849-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:03:59.849-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:03:59.849-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:03:59.922-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:03:59.922-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:03:59.934-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:04:00.000-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:00.015-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
PipelineManager 2017-04-25T13:04:00.323-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries 
PipelineManager 2017-04-25T13:04:00.323-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:04:00.323-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:00.323-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
PipelineRuntimeCtx 2017-04-25T13:04:00.323-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:04:00.323-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:04:00.323-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:04:00.323-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:04:00.323-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:00.325-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:04:00.325-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:00.323905911 -0500 CDT
GenericSupervisor 2017-04-25T13:04:00.325-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:00.325-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:04:00.325-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:00.325-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:04:00.325-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:00.325-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:04:00.325-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-518733274 has been stopped
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-518733274 is stopped
PipelineManager 2017-04-25T13:04:00.326-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
PipelineManager 2017-04-25T13:04:00.326-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:03:50.306168369-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
PipelineManager 2017-04-25T13:04:00.326-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-518733274
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:00.326-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:04:00.326-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:00.326-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:04:00.326-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:04:00.326-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:04:00.326-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:00.326-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:04:00.326-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:00.326-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:00.326467757 -0500 CDT
DcpNozzle 2017-04-25T13:04:00.326-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:04:00.326-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:00.326648614 -0500 CDT
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:00.326-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:04:00.327-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:04:00.327-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:04:00.327-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:04:00.327-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:04:00.327-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:04:00.327-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:00.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:00.327-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:04:00.327-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:00.327-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:04:00.329-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:00.329-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:00.329327488 -0500 CDT
DcpNozzle 2017-04-25T13:04:00.329-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:04:00.329-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:00.329-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:04:00 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:04:00 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:04:00.348-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:00.348-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:00.348915286 -0500 CDT
DcpNozzle 2017-04-25T13:04:00.348-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:04:00.348-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:00.349-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:04:00.403-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:00.403-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:04:00.403-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:04:00.529-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:00.529-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:04:00.529-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:00.529-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:04:00.529-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:00.530-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:04:00.530-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:04:00.530-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:04:00.530-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
GenericSupervisor 2017-04-25T13:04:00.532-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:04:00.532-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
GenericSupervisor 2017-04-25T13:04:00.532-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:00.532-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:00.532-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:00.532-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:00.532-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:04:00.532-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
CheckpointManager 2017-04-25T13:04:00.532-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
CapiUtils 2017-04-25T13:04:00.584-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:00.584-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:00.584-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:04:00.585-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
StatisticsManager 2017-04-25T13:04:00.585-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:04:00.585-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
XDCRFactory 2017-04-25T13:04:00.585-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:04:00.586-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498 is constructed, start it
GenericPipeline 2017-04-25T13:04:00.587-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[104 130 167 79 57 2 27 39 58 121 131 148 155 37 82 30 87 22 86 53 129 35 156 110 12 32 48 158 6 75 160 20 42 103 25 135 138 72 67 13 126 145 120 24 98 142 10 114 21 52 111 113 0 154 152 55 69 89 105 139 26 109 71 157 64 28 43 91 106 153 163 5 66 159 44 17 18 141 4 40 61 93 123 16 49 84 127 161 56 59 70 73 108 119 168 14 97 78 133 128 117 170 7 115 90 101 88 11 46 122 9 118 124 143 144 166 15 164 125 1 134 151 81 77 99 107 3 96 74 100 149 33 83 116 146 51 132 85 38 54 162 23 102 136 150 41 137 147 76 92 36 80 112 34 94 45 29 68 165 19 60 62 31 47 8 140 169 63 65 95 50] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[174 338 243 244 317 239 280 234 272 187 303 212 225 259 286 307 334 274 302 175 219 223 326 269 300 182 249 316 331 192 205 267 276 313 323 198 218 305 309 179 253 330 283 321 335 337 217 193 237 271 184 206 235 242 278 200 275 297 203 204 310 251 324 213 258 265 288 281 328 289 178 188 199 339 194 202 211 260 292 308 173 257 270 290 312 186 299 264 207 214 261 171 320 190 311 191 197 329 189 215 277 318 226 254 293 216 279 185 196 301 319 296 304 332 177 229 245 230 231 236 314 208 238 241 268 298 195 282 181 220 227 247 291 340 180 210 266 273 221 333 176 228 233 284 306 295 201 222 262 315 252 256 263 287 325 183 240 246 322 327 224 172 209 250 255 294 336 232 285 248]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[74 96 33 100 149 51 83 116 146 85 132 23 38 54 162 41 102 136 150 76 137 147 36 92 34 80 112 45 94 19 29 68 165 31 60 62 8 47 63 140 169 50 65 95 79 104 130 167 2 57 27 155 37 39 58 121 131 148 82 22 30 87 53 86 35 129 110 156 6 12 32 48 158 20 75 160 25 42 103 72 135 138 13 67 120 126 145 10 24 98 142 114 0 21 52 111 113 152 154 26 55 69 89 105 139 109 64 71 157 163 5 28 43 91 106 153 44 66 159 4 17 18 141 16 40 61 93 123 49 56 84 127 161 168 14 59 70 73 108 119 78 97 128 133 7 117 170 115 88 90 101 9 11 46 122 15 118 124 143 144 166 125 164 1 81 134 151 3 77 99 107] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[291 220 227 247 340 180 210 266 273 221 333 233 284 176 228 295 306 201 222 262 315 287 325 252 256 263 246 322 183 240 327 224 294 336 172 209 250 255 232 285 248 174 338 239 243 244 317 234 280 272 187 212 303 225 334 259 286 307 175 274 302 219 223 326 269 300 182 249 316 331 276 313 192 205 267 323 198 218 305 309 179 253 330 283 321 217 335 337 271 193 237 184 206 235 242 278 275 297 200 203 204 310 251 324 288 213 258 265 281 289 328 339 178 188 199 211 260 194 202 173 292 308 186 257 270 290 312 299 264 261 207 214 171 190 320 311 191 197 329 318 189 215 277 226 254 293 216 279 301 319 185 196 296 304 332 177 229 245 230 231 236 314 195 208 238 241 268 298 282 181]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 optimistic_replication_threshold:256 replication_type:capi filter_expression: target_nozzle_per_node:2 log_level:Info stats_interval:1000 failure_restart_interval:10 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:04:00.587-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498...
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:04:00.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:04:00.594-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:00.594-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:04:00.594-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:04:00.594-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:04:00.594-05:00 [INFO] Checkpointing starts in 1802.452 sec
PipelineRuntimeCtx 2017-04-25T13:04:00.594-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:04:00.594-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:04:00.594-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:04:00.594-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:04:00.595-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:04:00.595-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:04:00.595-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:04:00.595-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:04:00.596-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:04:00.616-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:04:00.616-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:04:00.616-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:04:00.636-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:04:00.637-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:04:00.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:04:00.647-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:00.647-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:00.647-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:04:00.649-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:04:00.650-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:04:00.650-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:00.650-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:04:00.650-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:04:00.650-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:04:00.650-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:04:00.667-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:04:00.667-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:04:00.667-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:04:00.672-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:04:00.682-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:00.682-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:00.789-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:04:00.789-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:00.789-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:04:00.789-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:04:00.789-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:04:00.789-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:04:00.789-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498 is started----------
PipelineManager 2017-04-25T13:04:00.790-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:04:00.790-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:04:00.790-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:04:00.823-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:04:01.095-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:04:01.096-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:04:01.096-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:04:01.096-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:04:01 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:04:01.229-05:00 [INFO] Send batch count=49
2017/04/25 13:04:01 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:04:01.265-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:01.269-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:01.274-05:00 [INFO] Send batch count=121
2017/04/25 13:04:01 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:04:01.309-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:01.309-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:01.332-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:04:01.336-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:04:01 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 119 successful
CapiNozzle 2017-04-25T13:04:01.481-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:04:01 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 322 successful
CapiNozzle 2017-04-25T13:04:01.508-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:04:01 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 326 successful
CapiNozzle 2017-04-25T13:04:01.611-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:01.627-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:04:01.649-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
StatisticsManager 2017-04-25T13:04:01.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:04:01 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 128 successful
CapiNozzle 2017-04-25T13:04:01.683-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
2017/04/25 13:04:01 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 56 successful
CapiNozzle 2017-04-25T13:04:01.899-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:04:01 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:04:01.924-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:04:01.944-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:01.945-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:04:01.945-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:04:01.945-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:04:01.945-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
PipelineManager 2017-04-25T13:04:01.954-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:04:01.954-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:04:01.956-05:00 [INFO] err_list=[{"time":"2017-04-25T13:04:01.956669337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:04:01.956-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:04:01 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:04:01 UPR_STREAMREQ for vb 258 successful
CapiNozzle 2017-04-25T13:04:02.036-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:02.036-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:02.067-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:04:02 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 274 successful
CapiNozzle 2017-04-25T13:04:02.134-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:02.134-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:04:02.134-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:04:02.134-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:02.134-05:00 [INFO] Send batch count=500
2017/04/25 13:04:02 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 85 successful
CapiNozzle 2017-04-25T13:04:02.197-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:02.197-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:04:02 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 276 successful
CapiNozzle 2017-04-25T13:04:02.215-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:04:02 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 292 successful
CheckpointManager 2017-04-25T13:04:02.367-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
CheckpointManager 2017-04-25T13:04:02.367-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:04:02 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:04:02.378-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:02.396-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:04:02 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:04:02 UPR_STREAMREQ for vb 197 successful
DcpNozzle 2017-04-25T13:04:02.458-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:04:02.495-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:04:02.601-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:04:02.601-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498 {"Errors": "[{\"time\":\"2017-04-25T13:04:01.956669337-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 78, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 14780, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 78, "set_filtered": 0, "set_received_from_dcp": 14780, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:04:02.619-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:02.637-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:04:02.808-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:02.832-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:04:02.960-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:02.975-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:04:03.236-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:03.236-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:04:03.236-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:04:03.236-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:03.236-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:03.354-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:03.354-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:03.372-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:04:03.417-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:03.417-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:04:03.417-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:04:03.417-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:03.417-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:03.497-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:03.497-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:03.521-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:04:03.849-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:03.861-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:04:03.949-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:03.988-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:04:04.180-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:04.196-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:04:04.380-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:04.393-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:04:04.573-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:04.573-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:04:04.573-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:04:04.573-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:04.573-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:04.626-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:04.626-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:04.654-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:04:04.748-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:04.748-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:04:04.748-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:04:04.748-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:04.748-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:04.832-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:04.832-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:04.845-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:04:04.958-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:04.980-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:04:05.272-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:05.283-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:04:05.495-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:05.532-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:04:05.727-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:05.750-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:04:05.833-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:05.834-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:04:05.834-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:04:05.834-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:05.834-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:05.901-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:05.901-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:05.915-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:04:06.255-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:06.255-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:06.255-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:06.255-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:06.255-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:06.353-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:06.353-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:06.371-05:00 [INFO] Retrying update docs for vb 309 for the 1th time
CapiNozzle 2017-04-25T13:04:06.504-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:06.524-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:04:06.949-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:04:06.975-05:00 [INFO] Retrying update docs for vb 309 for the 2th time
CapiNozzle 2017-04-25T13:04:07.349-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:07.371-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:04:07.503-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:04:07.520-05:00 [INFO] Retrying update docs for vb 309 for the 3th time
PipelineManager 2017-04-25T13:04:07.769-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:01.956669337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]}
]
CapiNozzle 2017-04-25T13:04:07.775-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:07.775-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:04:07.775-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:04:07.775-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:07.775-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:07.908-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:07.908-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:07.921-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:04:07.930-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:04:07.930-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 309 after 3 retries
CapiNozzle 2017-04-25T13:04:07.930-05:00 [ERROR] Raise error condition batch update docs failed for vb 309 after 3 retries
GenericSupervisor 2017-04-25T13:04:07.930-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:07.930-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:08.005-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:08.006-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
ReplicationManager 2017-04-25T13:04:08.035-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:04:08.043-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
CapiNozzle 2017-04-25T13:04:08.339-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:08.370-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:04:08.502-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:04:08.517-05:00 [INFO] Retrying update docs for vb 208 for the 2th time
CapiNozzle 2017-04-25T13:04:08.753-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:08.797-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:04:08.912-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:04:08.927-05:00 [INFO] Retrying update docs for vb 208 for the 3th time
CapiNozzle 2017-04-25T13:04:09.154-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:09.154-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:04:09.154-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:04:09.154-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:09.154-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:09.219-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:09.219-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:09.230-05:00 [INFO] Retrying update docs for vb 6 for the 1th time
CapiNozzle 2017-04-25T13:04:09.335-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:04:09.335-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 208 after 3 retries
CapiNozzle 2017-04-25T13:04:09.335-05:00 [ERROR] Raise error condition batch update docs failed for vb 208 after 3 retries
GenericSupervisor 2017-04-25T13:04:09.335-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:09.335-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:09.389-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 212
CapiNozzle 2017-04-25T13:04:09.476-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:09.476-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:09.487-05:00 [INFO] Retrying update docs for vb 212 for the 1th time
CapiNozzle 2017-04-25T13:04:09.639-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:04:09.650-05:00 [INFO] Retrying update docs for vb 6 for the 2th time
CapiNozzle 2017-04-25T13:04:10.037-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:04:10.047-05:00 [INFO] Retrying update docs for vb 212 for the 2th time
CapiNozzle 2017-04-25T13:04:10.207-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:04:10.218-05:00 [INFO] Retrying update docs for vb 6 for the 3th time
GenericSupervisor 2017-04-25T13:04:10.594-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:04:10.594-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:04:10.595-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:04:10.759-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:04:10.769-05:00 [INFO] Retrying update docs for vb 212 for the 3th time
ToplogyChangeDetector 2017-04-25T13:04:10.824-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:04:10.992-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:04:10.992-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 6 after 3 retries
CapiNozzle 2017-04-25T13:04:10.992-05:00 [ERROR] Raise error condition batch update docs failed for vb 6 after 3 retries
GenericSupervisor 2017-04-25T13:04:10.992-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:10.992-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:11.093-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:11.094-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:11.104-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:04:11.330-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:04:11.330-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 212 after 3 retries
CapiNozzle 2017-04-25T13:04:11.330-05:00 [ERROR] Raise error condition batch update docs failed for vb 212 after 3 retries
GenericSupervisor 2017-04-25T13:04:11.330-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:11.330-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:11.410-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:11.410-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:11.421-05:00 [INFO] Retrying update docs for vb 211 for the 1th time
CapiNozzle 2017-04-25T13:04:11.459-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:04:11.471-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:04:11.879-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:04:11.891-05:00 [INFO] Retrying update docs for vb 211 for the 2th time
PipelineManager 2017-04-25T13:04:11.956-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:04:11.957-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:04:11.957-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:11.957-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
PipelineRuntimeCtx 2017-04-25T13:04:11.957-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:04:11.957-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:11.957-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:04:11.957-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:11.957088794 -0500 CDT
GenericSupervisor 2017-04-25T13:04:11.957-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:11.957-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:04:11.957-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:11.957-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:04:11.957-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:11.957-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:04:11.957-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:04:11.957-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:04:11.957-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:04:11.957-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-530079498 has been stopped
GenericPipeline 2017-04-25T13:04:11.957-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-530079498 is stopped
PipelineManager 2017-04-25T13:04:11.957-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:04:11.957-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:01.956669337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:04:11.957-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:11.958-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
GenericPipeline 2017-04-25T13:04:11.958-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
GenericPipeline 2017-04-25T13:04:11.958-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
GenericPipeline 2017-04-25T13:04:11.958-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-530079498
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:11.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:04:11.958-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:11.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:04:11.958-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:11.959-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:04:11.959-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:11.959-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:04:11.959-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:11.959-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:04:11.959-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:11.959-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:11.959-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:04:11.959-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:11.95925527 -0500 CDT
DcpNozzle 2017-04-25T13:04:11.959-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:04:11.959-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:11.95951578 -0500 CDT
CapiNozzle 2017-04-25T13:04:11.960-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:04:11.960-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:11.960-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:04:11.960-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:11.960-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:04:11.960-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:11.960-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:11.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:11.961-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:04:11.961-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:11.961-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:04:11.963-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:11.963-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:11.963-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:11.96376085 -0500 CDT
DcpNozzle 2017-04-25T13:04:11.963-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:11.963774762 -0500 CDT
DcpNozzle 2017-04-25T13:04:11.963-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:04:11.963-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:04:11.963-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:11.963-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:04:11.963-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:11.963-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:04:11 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:04:11 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:04:12.050-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:12.050-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:04:12.050-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:04:12.136-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:12.137-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:04:12.137-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:12.137-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:04:12.137-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:12.137-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:04:12.137-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:04:12.138-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:04:12.138-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
GenericSupervisor 2017-04-25T13:04:12.140-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:04:12.140-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
GenericSupervisor 2017-04-25T13:04:12.140-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:12.140-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:12.140-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:12.140-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:12.140-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:04:12.140-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
CheckpointManager 2017-04-25T13:04:12.140-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
CapiUtils 2017-04-25T13:04:12.179-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:12.179-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:12.179-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:04:12.180-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
StatisticsManager 2017-04-25T13:04:12.180-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:04:12.180-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
XDCRFactory 2017-04-25T13:04:12.181-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:04:12.181-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575 is constructed, start it
GenericPipeline 2017-04-25T13:04:12.182-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[3 131 34 74 87 159 14 49 58 167 40 169 32 80 137 153 1 132 151 53 63 64 23 38 43 44 61 161 26 72 92 102 20 30 75 152 28 95 148 13 31 25 122 111 17 81 2 139 9 45 79 103 97 117 133 146 42 85 110 126 10 19 41 84 91 129 130 120 158 37 155 48 68 11 56 135 149 39 96 128 136 160 107 12 60 62 157 22 115 170 140 98 123 27 59 70 121 162 46 109 112 116 166 7 71 113 119 5 54 138 16 47 76 89 147 150 4 36 104 6 35 100 163 29 86 143 154 168 55 106 141 50 65 77 82 142 144 125 78 94 90 127 145 18 52 108 124 0 156 8 114 21 83 118 66 73 88 101 51 24 69 165 33 93 134 164 15 57 67 105 99] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[213 244 237 245 329 330 206 266 299 263 280 296 310 226 227 276 178 236 238 253 282 307 208 273 195 254 225 239 285 209 327 331 242 281 303 246 291 309 325 196 215 234 232 207 260 277 293 256 274 298 312 194 233 290 295 326 211 261 283 333 294 217 279 199 216 258 305 323 278 197 314 320 324 189 231 249 193 201 214 297 218 184 308 198 228 243 271 316 284 269 204 224 230 287 255 301 306 210 340 240 270 292 313 203 205 250 272 179 180 186 200 235 172 212 257 300 191 304 336 182 259 302 267 202 264 275 174 181 188 311 219 173 183 247 321 252 315 318 175 229 187 223 317 190 248 171 265 319 220 289 332 176 192 286 251 288 338 177 185 222 262 339 221 241 268 322 328 337 334 335]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[36 104 4 35 100 163 6 29 143 154 168 86 106 141 55 65 77 82 142 144 50 125 78 94 127 145 90 52 108 124 18 156 0 114 8 83 118 21 73 66 101 88 51 69 165 24 93 134 164 33 57 67 105 15 99 131 3 74 87 159 34 49 58 167 14 169 40 80 137 153 32 132 151 1 63 64 53 38 43 44 61 161 23 72 92 102 26 30 75 152 20 95 148 28 31 13 122 25 111 81 17 139 2 45 79 103 9 117 133 146 97 85 110 126 42 19 41 10 91 129 130 84 158 120 155 37 68 48 56 135 149 11 96 128 136 160 39 107 60 62 157 12 115 170 22 140 123 98 59 27 121 162 70 109 112 116 166 46 7 113 119 71 54 138 5 47 76 89 147 150 16] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[179 180 186 200 235 172 212 257 300 191 304 336 182 259 302 267 202 264 275 174 181 188 311 219 173 183 247 321 252 315 318 175 229 187 223 317 190 248 171 265 319 220 289 332 176 192 286 251 288 338 177 185 222 262 339 221 241 268 322 337 328 334 335 213 244 237 245 329 330 206 266 299 263 280 296 310 226 227 276 178 236 238 253 282 307 208 273 195 254 225 239 285 327 331 209 242 281 303 246 291 309 325 196 215 234 232 207 260 277 293 256 274 298 312 194 233 290 295 326 211 261 283 333 294 217 279 199 216 258 305 323 278 197 314 320 324 189 231 249 201 214 297 193 218 308 184 198 228 243 271 316 284 269 204 224 230 287 255 301 306 210 340 240 270 313 292 205 250 272 203]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 stats_interval:1000 replication_type:capi active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 log_level:Info]
CheckpointManager 2017-04-25T13:04:12.182-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575...
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:12.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:04:12.193-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:12.193-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:04:12.193-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:04:12.193-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:04:12.193-05:00 [INFO] Checkpointing starts in 1802.144 sec
PipelineRuntimeCtx 2017-04-25T13:04:12.193-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:04:12.193-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:04:12.194-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:04:12.194-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:04:12.198-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:04:12.198-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:04:12.198-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:04:12.199-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:04:12.203-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:04:12.230-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiUtils 2017-04-25T13:04:12.234-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:12.234-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:12.234-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:04:12.236-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:04:12.236-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:04:12.248-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:04:12.248-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:04:12.262-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:04:12.262-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:04:12.286-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:04:12.336-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:04:12.336-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:12.336-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:04:12.336-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:04:12.336-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:04:12.337-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:04:12.364-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:04:12.364-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:04:12.364-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:04:12.383-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:12.383-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:04:12.398-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:04:12.542-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:04:12.542-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:12.542-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:04:12.562-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:04:12.562-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:04:12.562-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:04:12.562-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575 is started----------
PipelineManager 2017-04-25T13:04:12.562-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:04:12.563-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:04:12.563-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:04:12.706-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:04:12.706-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:04:12.706-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:04:12.706-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:04:12 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:04:12.818-05:00 [INFO] Send batch count=123
CapiNozzle 2017-04-25T13:04:12.819-05:00 [INFO] Send batch count=99
CapiNozzle 2017-04-25T13:04:12.849-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:12.849-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:12.870-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:12.870-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:04:12 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 6 successful
CapiNozzle 2017-04-25T13:04:12.886-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:04:12.896-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:04:12 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:04:12 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 219 successful
CapiNozzle 2017-04-25T13:04:13.268-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:04:13 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:04:13 UPR_STREAMREQ for vb 222 successful
StatisticsManager 2017-04-25T13:04:13.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:04:13.307-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:04:14.745-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:04:14 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 133 successful
StatisticsManager 2017-04-25T13:04:14.819-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:04:14.819-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:04:14 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:04:14 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 240 successful
CapiNozzle 2017-04-25T13:04:15.199-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:15.232-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:04:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 45 successful
StatisticsManager 2017-04-25T13:04:15.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:04:15.279-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:04:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 243 successful
CapiNozzle 2017-04-25T13:04:15.306-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
2017/04/25 13:04:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 258 successful
CapiNozzle 2017-04-25T13:04:15.505-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:15.505-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:15.505-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:15.505-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:04:15.505-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:04:15.508-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:04:15.508-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:04:15.511-05:00 [INFO] err_list=[{"time":"2017-04-25T13:04:15.511047798-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:04:15.511-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:04:15 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 260 successful
CapiNozzle 2017-04-25T13:04:15.585-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:15.585-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:15.606-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:15.606-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:04:15.606-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:04:15.606-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:15.606-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:15.611-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:04:15 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 274 successful
CapiNozzle 2017-04-25T13:04:15.705-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:15.705-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:15.724-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:04:15 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 287 successful
CapiNozzle 2017-04-25T13:04:15.896-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CheckpointManager 2017-04-25T13:04:15.907-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
CheckpointManager 2017-04-25T13:04:15.907-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:04:15.925-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:04:15 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:04:15 UPR_STREAMREQ for vb 296 successful
DcpNozzle 2017-04-25T13:04:16.042-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:04:16.069-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:04:16.087-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:16.101-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
StatisticsManager 2017-04-25T13:04:16.205-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:04:16.205-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575 {"Errors": "[{\"time\":\"2017-04-25T13:04:15.511047798-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 86, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 34220, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 86, "set_filtered": 0, "set_received_from_dcp": 34220, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:04:16.233-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:16.269-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:04:16.458-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:16.474-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:04:16.620-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:16.620-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:16.620-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:16.620-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:16.620-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:16.706-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:16.706-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:16.721-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:04:16.782-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:16.783-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:04:16.783-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:04:16.783-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:16.783-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:16.843-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:16.843-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:16.854-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:04:17.081-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:17.102-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:04:17.190-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:17.207-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:04:17.517-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:17.545-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:04:17.693-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:17.714-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:04:17.949-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:17.949-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:04:17.949-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:04:17.949-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:17.949-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:18.009-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:18.009-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:18.052-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:04:18.058-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:18.058-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:04:18.058-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:04:18.058-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:18.058-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:18.100-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:18.100-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:18.113-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:04:18.474-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:04:18.488-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:04:18.734-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:18.750-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:04:18.954-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:04:18.992-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:04:19.111-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:19.124-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:04:19.429-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:04:19.429-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:04:19.429-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:04:19.429-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:19.429-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:19.491-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:19.491-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:19.514-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:04:19.611-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:19.612-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:04:19.612-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:04:19.612-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:19.612-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:19.717-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:19.717-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:19.746-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:04:20.064-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:20.081-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:04:20.241-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:04:20.263-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:04:20.712-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:20.730-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:04:20.874-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:04:20.895-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:04:21.464-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:04:21.464-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:04:21.464-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:04:21.464-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:21.464-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:21.602-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:21.602-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:21.621-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:04:21.694-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:04:21.694-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:04:21.694-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:04:21.694-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:21.694-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:21.809-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:21.809-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:21.825-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:04:22.083-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:04:22.107-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
GenericSupervisor 2017-04-25T13:04:22.203-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:04:22.203-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:04:22.204-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:04:22.478-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:22.499-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
ToplogyChangeDetector 2017-04-25T13:04:22.529-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:04:22.674-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:04:22.685-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
PipelineManager 2017-04-25T13:04:22.706-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:15.511047798-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:04:22.859-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:22.870-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:04:23.022-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:04:23.022-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:04:23.023-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:04:23.023-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:23.023-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:23.094-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:23.095-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:23.105-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:04:23.293-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:23.293-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:04:23.293-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:04:23.293-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:23.293-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:23.367-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:23.367-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:23.379-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:04:23.709-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:04:23.720-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:04:23.974-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:04:23.985-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:04:24.402-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:04:24.413-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:04:24.553-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:04:24.564-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:04:24.777-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:04:24.777-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:04:24.777-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:04:24.777-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:24.777-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:24.871-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:24.872-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:24.883-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:04:24.962-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:04:24.962-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:04:24.962-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:04:24.962-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:24.962-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:25.018-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:25.018-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:25.029-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:04:25.306-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:04:25.317-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:04:25.427-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:25.438-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
PipelineManager 2017-04-25T13:04:25.511-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:04:25.511-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:04:25.511-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:25.511-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
PipelineRuntimeCtx 2017-04-25T13:04:25.511-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:04:25.511-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:25.511-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:04:25.511-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:25.511543845 -0500 CDT
GenericSupervisor 2017-04-25T13:04:25.511-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:25.511-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:04:25.511-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:25.511-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:04:25.511-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:25.511-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:04:25.511-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:04:25.511-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:04:25.511-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:04:25.511-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-137736575 has been stopped
GenericPipeline 2017-04-25T13:04:25.511-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-137736575 is stopped
PipelineManager 2017-04-25T13:04:25.511-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:04:25.512-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:15.511047798-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:04:25.512-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:25.512-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
GenericPipeline 2017-04-25T13:04:25.512-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
GenericPipeline 2017-04-25T13:04:25.512-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
GenericPipeline 2017-04-25T13:04:25.512-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-137736575
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:25.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:04:25.512-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:25.512-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:04:25.513-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:25.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:04:25.513-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:25.513-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:04:25.513-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:25.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:04:25.513-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:25.513-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:25.513-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:25.513074853 -0500 CDT
DcpNozzle 2017-04-25T13:04:25.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:04:25.514-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:25.513451903 -0500 CDT
DcpNozzle 2017-04-25T13:04:25.514-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:04:25.514-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:04:25.514-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:25.514-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:04:25.514-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:25.515-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:04:25.515-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:25.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:25.515-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:04:25.515-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:25.515-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:04:25.520-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:25.520-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:25.520-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:25.520130522 -0500 CDT
DcpNozzle 2017-04-25T13:04:25.520-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:25.520149299 -0500 CDT
DcpNozzle 2017-04-25T13:04:25.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:04:25.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:04:25.520-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:25.520-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:04:25.520-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:25.520-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:04:25 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:04:25 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:04:25 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:04:25 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:04:25 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:04:25 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:04:25 Stream Closed for vb 2 StreamEnd simulated
XDCRFactory 2017-04-25T13:04:25.597-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:25.597-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:04:25.597-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:04:25.738-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:25.739-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:04:25.740-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:25.740-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:04:25.740-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:25.740-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:04:25.740-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:04:25.741-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:04:25.741-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
GenericSupervisor 2017-04-25T13:04:25.744-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:04:25.744-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
GenericSupervisor 2017-04-25T13:04:25.744-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:25.744-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:25.744-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:25.744-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:25.744-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:04:25.745-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
CheckpointManager 2017-04-25T13:04:25.745-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
CapiUtils 2017-04-25T13:04:25.813-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:25.813-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:25.813-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:04:25.814-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
StatisticsManager 2017-04-25T13:04:25.815-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:04:25.815-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
XDCRFactory 2017-04-25T13:04:25.815-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:04:25.815-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387 is constructed, start it
GenericPipeline 2017-04-25T13:04:25.817-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[216 275 292 325 208 309 226 227 262 173 232 233 270 314 298 255 209 220 222 250 263 196 178 235 241 320 324 257 274 225 322 190 177 334 175 224 234 313 339 201 171 311 219 244 315 335 188 193 245 319 258 267 181 284 295 310 286 305 330 185 312 316 297 282 210 280 328 199 252 268 180 264 308 230 237 247 265 202 174 296 207 229 323 251 273 290 340 176 192 203 271 276 304 246 189 261 256 272 289 285 213 242 306 331 221 239 198 238 281 223 287 206 200 269 332 299 183 182 184 318 204 217 291 321 336 187 194 228 293 326 259 266 277 283 197 253 278 288 307 329 214 248 218 211 243 303 317 338 212 186 260 337 300 195 236 179 279 302 333 215 240 327 231 249 254 294 172 205 301 191] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[93 87 128 6 165 144 28 47 125 52 105 114 160 36 92 115 62 137 32 45 11 33 121 0 2 136 23 40 82 100 146 155 84 101 83 152 17 65 103 34 51 123 162 10 97 147 46 107 153 154 157 111 69 139 63 148 48 42 9 37 74 16 43 86 120 39 80 96 98 141 67 70 166 31 73 72 13 60 4 150 90 133 68 94 1 29 140 143 159 19 22 116 145 158 71 88 112 118 170 27 85 50 122 134 95 117 5 41 81 15 26 76 104 35 57 79 132 138 38 54 149 163 168 44 89 20 106 108 131 7 59 91 127 24 58 61 167 109 135 156 55 66 110 126 56 77 130 169 3 8 151 164 49 102 119 142 161 30 78 113 124 129 25 53 18 99 21 64 75 12 14]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[282 297 199 210 280 328 252 268 180 264 308 202 230 237 247 265 174 296 207 229 323 251 273 290 340 176 192 203 271 276 304 189 246 261 256 272 289 285 213 242 239 306 331 221 198 238 281 206 223 287 200 269 332 183 299 182 184 318 204 217 291 321 336 187 194 228 266 293 326 259 253 277 283 197 278 288 307 329 214 218 248 211 243 303 338 317 212 186 260 337 300 195 236 179 279 302 333 215 240 327 231 249 254 294 172 205 301 191 208 216 275 292 325 226 309 227 262 173 232 233 270 314 255 298 196 209 220 222 250 263 178 235 241 274 320 324 257 190 225 322 177 334 175 224 234 313 339 201 171 311 219 244 315 335 188 193 245 319 258 267 181 284 295 310 286 305 330 185 312 316] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[157 111 139 69 148 63 48 37 42 9 43 74 16 80 86 120 39 70 96 98 141 67 73 166 31 60 72 13 4 133 150 90 29 68 94 1 22 140 143 159 19 88 116 145 158 71 85 112 118 170 27 50 117 122 134 95 5 26 41 81 15 57 76 104 35 54 79 132 138 38 89 149 163 168 44 106 20 131 108 7 58 59 91 127 24 167 61 66 109 135 156 55 110 77 126 56 8 130 169 3 102 151 164 49 78 119 142 161 30 124 113 53 129 25 99 18 14 21 64 75 12 93 87 128 165 6 47 144 28 125 52 92 105 114 160 36 115 45 62 137 32 33 11 2 121 0 40 136 23 100 82 101 146 155 84 65 83 152 17 51 103 34 97 123 162 10 107 147 46 154 153]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 replication_type:capi failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression:]
CheckpointManager 2017-04-25T13:04:25.817-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387...
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:04:25.838-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:04:25.838-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:25.838-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:04:25.838-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:04:25.838-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:04:25.838-05:00 [INFO] Checkpointing starts in 1801.558 sec
PipelineRuntimeCtx 2017-04-25T13:04:25.838-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:04:25.838-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:04:25.838-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:04:25.839-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:04:25.842-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:04:25.842-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:04:25.842-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:04:25.842-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:04:25.843-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:04:25.881-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:04:25.881-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:04:25.881-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:04:25.904-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:04:25.905-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:04:25.916-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:04:25.940-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:04:25.945-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:25.945-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:25.945-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:04:25.947-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:04:25.947-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:25.947-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:04:25.947-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:04:25.947-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:04:25.947-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:04:25.974-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:04:25.974-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:04:25.975-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:04:25.996-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:04:25.996-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:25.996-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:26.148-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:04:26.148-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:26.148-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:04:26.172-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:04:26.172-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:04:26.172-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:04:26.172-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387 is started----------
PipelineManager 2017-04-25T13:04:26.172-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:04:26.172-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:04:26.172-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:04:26.197-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:04:26.425-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:04:26.425-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:04:26.426-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:04:26.426-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:04:26 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:04:26.486-05:00 [INFO] Send batch count=102
2017/04/25 13:04:26 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:04:26.536-05:00 [INFO] Send batch count=20
CapiNozzle 2017-04-25T13:04:26.553-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:26.553-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:04:26 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:04:26.571-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:04:26 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 215 successful
CapiNozzle 2017-04-25T13:04:26.879-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:04:26 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 328 successful
CapiNozzle 2017-04-25T13:04:26.895-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
StatisticsManager 2017-04-25T13:04:26.942-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:04:26 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:04:26 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 340 successful
CapiNozzle 2017-04-25T13:04:27.137-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:27.165-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:04:27 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 176 successful
CapiNozzle 2017-04-25T13:04:27.313-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:27.324-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:04:27.324-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:04:27.324-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:04:27.324-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:04:27.333-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:04:27.333-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:04:27.334-05:00 [INFO] err_list=[{"time":"2017-04-25T13:04:27.334564394-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:04:27.334-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:04:27 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 185 successful
CapiNozzle 2017-04-25T13:04:27.414-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:27.414-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:27.445-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:04:27 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 294 successful
CheckpointManager 2017-04-25T13:04:27.535-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
CheckpointManager 2017-04-25T13:04:27.535-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:04:27 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:04:27 UPR_STREAMREQ for vb 299 successful
DcpNozzle 2017-04-25T13:04:27.656-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:04:27.687-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:04:27.847-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:04:27.848-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387 {"Errors": "[{\"time\":\"2017-04-25T13:04:27.334564394-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 52, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 15009, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 52, "set_filtered": 0, "set_received_from_dcp": 15009, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:04:28.181-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:28.208-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:04:28.550-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:28.578-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:04:28.999-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:28.999-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:04:28.999-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:04:28.999-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:28.999-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:29.076-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:29.076-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:29.088-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:04:29.402-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:29.425-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:04:29.739-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:29.759-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:04:30.425-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:30.425-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:04:30.425-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:04:30.425-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:30.425-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:30.505-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:30.505-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:30.517-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:04:30.867-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:04:30.887-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:04:31.187-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:04:31.206-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:04:31.548-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:04:31.548-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:04:31.548-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:04:31.548-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:31.548-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:31.611-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:31.612-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:31.633-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:04:31.932-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:04:31.958-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:04:32.347-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:04:32.370-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:04:32.769-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:04:32.769-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:04:32.769-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:04:32.769-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:32.769-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:32.843-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:32.843-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:32.862-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:04:33.235-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:33.269-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:04:33.620-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:33.640-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:04:33.958-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:33.959-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:04:33.959-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:04:33.959-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:33.959-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:34.057-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:34.057-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:34.075-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:04:34.432-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:34.443-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:04:34.798-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:34.809-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:04:35.290-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:35.290-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:04:35.290-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:04:35.290-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:35.290-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:35.350-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:35.350-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:35.361-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
GenericSupervisor 2017-04-25T13:04:35.839-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:04:35.839-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:04:35.839-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:04:35.852-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:35.862-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
ToplogyChangeDetector 2017-04-25T13:04:36.193-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:04:36.266-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:36.277-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:04:36.556-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:04:36.567-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:04:36.626-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:36.626-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:04:36.626-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:04:36.626-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:36.626-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:36.630-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:36.641-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:04:36.680-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:36.691-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:04:36.718-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:36.718-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:36.720-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:36.720-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:36.720-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:36.720-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:36.720-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:36.729-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:04:36.798-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:36.798-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:36.809-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:04:37.115-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:04:37.126-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
PipelineManager 2017-04-25T13:04:37.335-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:04:37.335-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:04:37.335-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:37.335-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
PipelineRuntimeCtx 2017-04-25T13:04:37.335-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:04:37.335-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:04:37.335-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:04:37.335-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:04:37.335-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:37.335-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:04:37.335-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:37.335415471 -0500 CDT
GenericSupervisor 2017-04-25T13:04:37.335-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:37.335-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:04:37.335-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:37.335-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:04:37.335-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:37.335-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:04:37.335-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-740569387 has been stopped
GenericPipeline 2017-04-25T13:04:37.335-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-740569387 is stopped
PipelineManager 2017-04-25T13:04:37.335-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:04:37.336-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:27.334564394-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:04:37.336-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:37.336-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
GenericPipeline 2017-04-25T13:04:37.336-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
GenericPipeline 2017-04-25T13:04:37.336-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
GenericPipeline 2017-04-25T13:04:37.336-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-740569387
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:37.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:37.337-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:37.337-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:37.337-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:37.337-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:37.337-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:04:37.337-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:37.337-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:04:37.337-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:04:37.337-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:37.337-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:04:37.337-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:04:37.337-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:37.337-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:04:37.337-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:37.338-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:04:37.338-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:37.338-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:37.338-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:04:37.338-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:37.337862412 -0500 CDT
DcpNozzle 2017-04-25T13:04:37.338-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:04:37.338-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:37.338205298 -0500 CDT
CapiNozzle 2017-04-25T13:04:37.339-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:04:37.339-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:37.339-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:04:37.339-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:37.339-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:04:37.340-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:37.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:04:37.341-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:37.341-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:04:37.341-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:37.341-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:04:37.345-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:37.345-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:37.345470295 -0500 CDT
DcpNozzle 2017-04-25T13:04:37.345-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:04:37.345-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:37.345-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:04:37 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:04:37 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:04:37.367-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:37.367-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:37.367939544 -0500 CDT
DcpNozzle 2017-04-25T13:04:37.368-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:04:37.368-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:37.368-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:04:37.445-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:37.445-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:04:37.445-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:04:37.520-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:37.520-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:04:37.520-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:37.520-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:04:37.521-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:37.521-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:04:37.521-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:04:37.521-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:04:37.522-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
GenericSupervisor 2017-04-25T13:04:37.523-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:04:37.523-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
GenericSupervisor 2017-04-25T13:04:37.524-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:37.524-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:37.524-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:37.524-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:37.524-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:04:37.524-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
CheckpointManager 2017-04-25T13:04:37.524-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
CapiUtils 2017-04-25T13:04:37.550-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:37.550-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:37.550-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:04:37.551-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
StatisticsManager 2017-04-25T13:04:37.551-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:04:37.551-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
XDCRFactory 2017-04-25T13:04:37.551-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:04:37.551-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839 is constructed, start it
GenericPipeline 2017-04-25T13:04:37.552-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[30 76 117 21 157 155 156 17 97 100 116 73 7 82 107 110 38 79 140 142 159 61 111 125 130 133 81 135 170 120 18 15 41 43 58 12 138 77 90 103 126 132 57 163 2 127 52 145 147 121 53 56 95 109 51 137 93 119 42 50 35 37 69 91 106 152 8 27 149 25 62 71 131 46 80 115 10 68 16 112 148 1 150 74 67 86 134 154 4 66 136 31 162 164 99 9 40 166 5 29 24 33 122 28 36 47 141 19 23 101 118 6 108 143 144 48 167 169 11 87 146 72 84 59 70 113 14 75 151 161 44 92 104 64 49 114 124 26 88 89 123 22 63 94 34 158 98 129 160 102 83 105 65 85 128 32 55 60 45 54 78 168 3 153 139 13 20 96 0 165 39] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[171 215 340 210 267 300 177 258 264 317 224 208 193 217 259 311 319 238 245 220 226 313 327 204 185 194 244 305 308 269 223 234 332 173 175 229 251 256 285 188 213 270 284 322 189 206 228 278 298 306 211 296 325 312 179 187 201 281 291 248 214 225 232 274 276 197 304 307 174 227 241 321 250 295 205 265 277 318 310 262 280 286 275 283 186 246 230 301 328 181 288 289 337 252 279 331 272 184 257 290 338 339 172 195 271 209 216 261 207 268 314 182 219 320 323 334 202 333 253 199 297 330 176 231 233 287 192 218 273 329 198 203 180 191 324 255 326 336 190 212 249 335 196 221 236 302 254 200 242 237 315 222 240 260 266 303 183 299 294 293 239 178 247 282 292 316 235 243 263 309]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[4 67 86 134 154 31 66 136 99 162 164 9 5 40 166 24 29 28 33 122 19 36 47 141 6 23 101 118 48 108 143 144 11 167 169 72 87 146 59 84 14 70 113 44 75 151 161 64 92 104 26 49 114 124 22 88 89 123 34 63 94 158 98 102 129 160 65 83 105 32 85 128 45 55 60 3 54 78 168 139 153 0 13 20 96 39 165 21 30 76 117 155 157 17 156 73 97 100 116 7 38 82 107 110 61 79 140 142 159 81 111 125 130 133 120 135 170 18 12 15 41 43 58 138 57 77 90 103 126 132 2 163 52 127 121 145 147 51 53 56 95 109 93 137 119 35 42 50 8 37 69 91 106 152 25 27 149 46 62 71 131 10 80 115 16 68 1 112 148 74 150] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[272 339 184 257 290 338 172 195 271 209 216 261 207 268 314 334 182 219 320 323 202 333 253 199 297 330 176 231 233 287 192 218 273 329 198 203 180 191 324 255 326 336 190 212 249 335 196 221 236 254 302 200 242 237 315 303 222 240 260 266 183 299 294 239 293 178 247 282 292 316 235 243 263 309 171 215 340 177 210 267 300 258 264 317 208 224 193 217 259 311 319 238 245 220 204 226 313 327 244 185 194 269 305 308 223 234 332 173 175 229 251 256 188 285 284 213 270 322 189 206 228 278 298 306 296 211 325 312 291 179 187 201 281 248 214 225 232 274 276 197 304 307 174 227 241 321 205 250 295 277 265 310 318 262 280 286 275 283 186 246 181 230 301 328 288 289 337 252 279 331]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[filter_expression: worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info replication_type:capi active:true checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 stats_interval:1000]
CheckpointManager 2017-04-25T13:04:37.552-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839...
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:37.557-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:04:37.557-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:37.557-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:04:37.557-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:04:37.557-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:04:37.557-05:00 [INFO] Checkpointing starts in 1803.069 sec
PipelineRuntimeCtx 2017-04-25T13:04:37.557-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:04:37.557-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:04:37.558-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:04:37.558-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:04:37.560-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:04:37.560-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:04:37.560-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:04:37.560-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:04:37.563-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:04:37.577-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:04:37.577-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:04:37.577-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:04:37.586-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:04:37.589-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:04:37.589-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:04:37.594-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:37.594-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:37.594-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:04:37.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:04:37.618-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:04:37.632-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:04:37.632-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:37.632-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:04:37.635-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:04:37.635-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:04:37.635-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:04:37.663-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:04:37.663-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:04:37.663-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:04:37.679-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:37.679-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:04:37.736-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:04:37.771-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:04:37.771-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:37.772-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:04:37.772-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:04:37.772-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:04:37.772-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:04:37.772-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839 is started----------
PipelineManager 2017-04-25T13:04:37.772-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:04:37.772-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:04:37.772-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:04:37.964-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:04:37.964-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:04:37.965-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:04:37.965-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
PipelineManager 2017-04-25T13:04:38.009-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:04:38 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 103 successful
CapiNozzle 2017-04-25T13:04:38.128-05:00 [INFO] Send batch count=181
CapiNozzle 2017-04-25T13:04:38.130-05:00 [INFO] Send batch count=97
2017/04/25 13:04:38 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:04:38.189-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:38.194-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:38.216-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:38.216-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:04:38 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:04:38.228-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:04:38.245-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:04:38 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 18 successful
CapiNozzle 2017-04-25T13:04:38.510-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:04:38 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 319 successful
CapiNozzle 2017-04-25T13:04:38.533-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:04:38 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 321 successful
StatisticsManager 2017-04-25T13:04:38.624-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:04:38 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 25 successful
CapiNozzle 2017-04-25T13:04:38.684-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:04:38 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 231 successful
CapiNozzle 2017-04-25T13:04:38.737-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:04:38.769-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:04:38 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 34 successful
CapiNozzle 2017-04-25T13:04:38.801-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
2017/04/25 13:04:38 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:04:38 UPR_STREAMREQ for vb 251 successful
CapiNozzle 2017-04-25T13:04:39.021-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:39.037-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:04:39.047-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:04:39.047-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:04:39.047-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:04:39.047-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:04:39.047-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
2017/04/25 13:04:39 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 156 successful
PipelineManager 2017-04-25T13:04:39.078-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:04:39.079-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:04:39.084-05:00 [INFO] err_list=[{"time":"2017-04-25T13:04:39.084156821-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:04:39.084-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:04:39 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 63 successful
CapiNozzle 2017-04-25T13:04:39.185-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:39.185-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:04:39 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:04:39.209-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:04:39 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 74 successful
CapiNozzle 2017-04-25T13:04:39.384-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:39.384-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:39.384-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:39.384-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:39.384-05:00 [INFO] Send batch count=500
2017/04/25 13:04:39 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 88 successful
CapiNozzle 2017-04-25T13:04:39.471-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:39.471-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:04:39 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 286 successful
CapiNozzle 2017-04-25T13:04:39.486-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:04:39 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 95 successful
CheckpointManager 2017-04-25T13:04:39.557-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
CheckpointManager 2017-04-25T13:04:39.557-05:00 [INFO] Done with SetVBTimestamps
StatisticsManager 2017-04-25T13:04:39.565-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:04:39.566-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839 {"Errors": "[{\"time\":\"2017-04-25T13:04:39.084156821-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 121, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 14801, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 121, "set_filtered": 0, "set_received_from_dcp": 14801, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

2017/04/25 13:04:39 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:04:39 UPR_STREAMREQ for vb 198 successful
DcpNozzle 2017-04-25T13:04:39.647-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:04:39.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:04:39.751-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:39.772-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:04:39.922-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:39.958-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:04:40.126-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:40.144-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:04:40.401-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:40.434-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:04:40.604-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:40.604-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:04:40.604-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:04:40.604-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:40.604-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:40.717-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:40.717-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:40.745-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:04:40.761-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:40.761-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:04:40.761-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:04:40.761-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:40.761-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:42.950-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:42.963-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:42.972-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:43.014-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:04:43.032-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:04:43.737-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:43.760-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:04:44.001-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:44.017-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:04:44.722-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:44.781-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:04:44.950-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:04:44.951-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:04:44.951-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:04:44.951-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:44.952-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:45.014-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:45.014-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:45.042-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:04:45.506-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:45.506-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:45.506-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:45.506-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:45.506-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:45.571-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:45.571-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:45.610-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:04:45.761-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:45.792-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:04:46.407-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:04:46.423-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:04:46.641-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:46.656-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:04:47.287-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:04:47.308-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
GenericSupervisor 2017-04-25T13:04:47.561-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:04:47.561-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:04:47.561-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:04:47.587-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:47.587-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:04:47.587-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:04:47.587-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:47.587-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:47.663-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:47.663-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:47.679-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
ToplogyChangeDetector 2017-04-25T13:04:47.745-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:04:48.124-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:04:48.124-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:04:48.124-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:04:48.124-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:48.124-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:48.305-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:48.305-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:48.319-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:04:48.397-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:48.422-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:04:48.952-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:04:48.972-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
PipelineManager 2017-04-25T13:04:49.088-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:04:49.088-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:04:49.088-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:49.088-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
PipelineRuntimeCtx 2017-04-25T13:04:49.088-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:04:49.088-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:49.089-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:04:49.089-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:49.088844511 -0500 CDT
GenericSupervisor 2017-04-25T13:04:49.089-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:49.089-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:04:49.089-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:49.089-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:04:49.089-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:04:49.089-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:04:49.089-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:04:49.089-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:04:49.089-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:04:49.089-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-521126839 has been stopped
GenericPipeline 2017-04-25T13:04:49.089-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-521126839 is stopped
PipelineManager 2017-04-25T13:04:49.089-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:04:49.089-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:39.084156821-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:04:49.089-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:04:49.089-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
GenericPipeline 2017-04-25T13:04:49.089-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
GenericPipeline 2017-04-25T13:04:49.089-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
GenericPipeline 2017-04-25T13:04:49.089-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-521126839
AsyncListener 2017-04-25T13:04:49.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:49.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:49.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:49.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:49.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:49.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:49.090-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
CapiNozzle 2017-04-25T13:04:49.090-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:04:49.090-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:04:49.090-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:04:49.090-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
GenericPipeline 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:04:49.090-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:04:49.090-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:04:49.090-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:04:49.090-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:04:49.090-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:49.090123212 -0500 CDT
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:04:49.090-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:49.090392714 -0500 CDT
DcpNozzle 2017-04-25T13:04:49.090-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:04:49.091-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:04:49.091-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:04:49.091-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:04:49.091-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
DcpNozzle 2017-04-25T13:04:49.091-05:00 [INFO] Ask uprfeed to close
CapiNozzle 2017-04-25T13:04:49.091-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:04:49.091-05:00 [INFO] releasing capi client
DcpNozzle 2017-04-25T13:04:49.091-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:49.091646632 -0500 CDT
DcpNozzle 2017-04-25T13:04:49.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:49.091-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:04:49.091-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:04:49.091-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:04:49.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:04:49.091-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:04:49.092-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:04:49.092-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:04:49.092664699 -0500 CDT
DcpNozzle 2017-04-25T13:04:49.092-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:04:49.092-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:04:49.092-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:04:49 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:04:49 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:04:49 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:04:49 Stream Closed for vb 2 StreamEnd simulated
XDCRFactory 2017-04-25T13:04:49.166-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:49.166-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:04:49.166-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:04:49.247-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:04:49.247-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:04:49.248-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:49.248-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:04:49.248-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:04:49.248-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:04:49.248-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:04:49.248-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:04:49.248-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
GenericSupervisor 2017-04-25T13:04:49.249-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:04:49.249-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
GenericSupervisor 2017-04-25T13:04:49.249-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:49.249-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:49.250-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:49.250-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:04:49.250-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:04:49.250-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
CheckpointManager 2017-04-25T13:04:49.250-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
CapiUtils 2017-04-25T13:04:49.317-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:49.318-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:49.318-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:04:49.318-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
StatisticsManager 2017-04-25T13:04:49.318-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:04:49.319-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
XDCRFactory 2017-04-25T13:04:49.319-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:04:49.319-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752 is constructed, start it
GenericPipeline 2017-04-25T13:04:49.319-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[215 222 305 190 287 299 220 224 319 191 246 279 280 331 241 253 320 226 221 276 189 183 195 238 302 339 175 250 266 237 306 171 184 225 283 186 271 294 228 245 328 199 212 336 198 247 182 270 288 335 192 227 235 274 242 243 206 211 230 188 234 284 298 308 316 304 293 282 307 194 239 309 229 240 268 290 317 187 203 201 207 218 289 314 255 277 196 217 326 301 180 256 338 176 273 174 232 248 265 321 216 249 311 329 177 236 260 267 285 315 334 297 312 214 291 313 272 281 286 251 252 202 208 262 263 332 193 327 233 303 323 269 296 261 324 179 219 330 337 213 275 295 340 200 258 325 209 322 178 197 204 259 185 210 278 181 310 244 172 205 264 292 300 318 231 254 333 173 223 257] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[60 100 106 131 14 19 37 17 139 120 130 149 142 42 126 163 81 111 147 54 63 80 90 156 33 36 107 83 114 158 138 1 55 99 166 89 101 152 11 49 86 134 9 98 116 13 21 40 85 133 162 30 62 151 46 113 161 15 41 102 31 45 146 10 67 109 75 153 24 88 92 48 155 141 144 20 29 104 5 8 70 51 53 122 4 38 50 56 64 119 3 12 16 87 128 169 0 71 79 108 160 44 91 95 167 7 115 143 68 74 154 32 135 65 127 129 140 159 23 28 52 66 76 164 112 150 2 47 78 165 168 170 39 58 132 34 84 148 157 35 93 125 97 118 59 145 22 27 43 94 72 103 73 96 6 77 105 82 136 69 117 57 110 124 18 61 123 137 25 26 121]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[120 130 149 142 42 126 163 81 111 147 54 63 80 90 33 36 107 156 83 114 158 1 55 99 138 89 101 152 166 11 49 86 134 9 98 116 13 21 40 85 133 30 62 151 162 46 113 161 15 41 102 31 45 10 67 109 146 75 24 88 92 153 48 155 141 20 29 104 144 5 8 70 53 122 4 38 50 51 64 119 3 12 16 56 128 169 0 71 79 87 108 160 44 91 95 7 115 143 167 68 74 154 32 135 65 129 140 159 23 28 52 127 66 76 164 112 150 2 47 78 165 168 39 58 132 170 34 84 148 35 93 125 157 97 118 145 22 27 43 59 94 72 103 73 96 6 77 105 82 136 69 117 57 110 124 18 61 123 25 26 121 137 100 106 131 14 19 37 60 17 139] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[246 279 280 331 253 320 226 241 276 189 221 195 238 302 339 175 183 250 266 237 306 171 184 225 283 271 294 186 245 328 199 212 228 336 247 182 198 288 270 335 192 227 235 274 243 206 211 230 242 234 284 298 308 188 316 293 304 282 307 194 309 229 239 268 290 317 187 203 240 201 207 218 289 314 277 255 196 217 326 301 180 256 338 273 174 176 248 265 321 232 216 249 311 329 236 260 267 285 177 315 334 297 214 291 312 313 281 286 251 252 272 208 262 263 332 193 202 233 303 327 269 296 323 324 261 179 219 330 337 275 213 340 200 258 295 325 322 178 209 204 259 197 210 278 181 185 244 310 205 264 292 300 172 318 254 333 231 173 223 257 215 222 305 190 287 299 224 319 191 220]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression: worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 replication_type:capi]
CheckpointManager 2017-04-25T13:04:49.320-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752...
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:04:49.343-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:04:49.343-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:04:49.343-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:04:49.343-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:04:49.343-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:04:49.343-05:00 [INFO] Checkpointing starts in 1804.98 sec
PipelineRuntimeCtx 2017-04-25T13:04:49.343-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:04:49.343-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:04:49.343-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:04:49.343-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:04:49.345-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:04:49.345-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:04:49.345-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:04:49.345-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:04:49.347-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:04:49.362-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:04:49.363-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:04:49.363-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:04:49.364-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:04:49.382-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
StatisticsManager 2017-04-25T13:04:49.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:04:49.387-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:04:49.387-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
ToplogyChangeDetector 2017-04-25T13:04:49.390-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:04:49.390-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:04:49.390-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:04:49.439-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:04:49.460-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:04:49.460-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:49.460-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:04:49.460-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:04:49.461-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:04:49.461-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:04:49.492-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:04:49.492-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:04:49.492-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:04:49.520-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:04:49.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:04:49.690-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:04:49.690-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:04:49.690-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:04:49.690-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:04:49.717-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:04:49.717-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:04:49.717-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:04:49.717-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:04:49.717-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:04:49.717-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:04:49.718-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752 is started----------
PipelineManager 2017-04-25T13:04:49.718-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:04:49.718-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:04:49.718-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:04:49 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:04:49.778-05:00 [INFO] Send batch count=3
CapiNozzle 2017-04-25T13:04:49.789-05:00 [ERROR] Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f302/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:04:49.789-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f302/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
2017/04/25 13:04:49 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 304 successful
CapiNozzle 2017-04-25T13:04:49.824-05:00 [INFO] Send batch count=54
CapiNozzle 2017-04-25T13:04:49.836-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:49.836-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:04:49.836-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:49.845-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:04:49 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 107 successful
CapiNozzle 2017-04-25T13:04:49.880-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:04:49.881-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:04:49 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 217 successful
CapiNozzle 2017-04-25T13:04:49.889-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:49.904-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:04:49 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 110 successful
CapiNozzle 2017-04-25T13:04:49.992-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:49.992-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:04:49.992-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:04:49.993-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:04:49.993-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:04:49 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:04:49 UPR_STREAMREQ for vb 222 successful
PipelineManager 2017-04-25T13:04:49.998-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:04:49.998-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:04:50.001-05:00 [INFO] err_list=[{"time":"2017-04-25T13:04:50.001726005-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:04:50.001-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:04:50 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 129 successful
CapiNozzle 2017-04-25T13:04:50.045-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:04:50.075-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
2017/04/25 13:04:50 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 231 successful
CapiNozzle 2017-04-25T13:04:50.093-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:50.093-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:50.113-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:04:50 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 149 successful
StatisticsManager 2017-04-25T13:04:50.368-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:04:50.369-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752 {"Errors": "[{\"time\":\"2017-04-25T13:04:50.001726005-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

2017/04/25 13:04:50 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 281 successful
CapiNozzle 2017-04-25T13:04:50.697-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:04:50.714-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
2017/04/25 13:04:50 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:04:50 UPR_STREAMREQ for vb 93 successful
CheckpointManager 2017-04-25T13:04:50.942-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
CheckpointManager 2017-04-25T13:04:50.942-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:04:50.961-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
DcpNozzle 2017-04-25T13:04:50.972-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:04:51.000-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:04:51 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:04:51 UPR_STREAMREQ for vb 98 successful
DcpNozzle 2017-04-25T13:04:51.139-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:04:51.221-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:04:51.221-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:04:51.221-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:04:51.221-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:51.221-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:51.296-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:51.296-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:51.314-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:04:51.678-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:51.691-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:04:52.079-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:04:52.096-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:04:52.492-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:04:52.492-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:04:52.492-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:04:52.492-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:52.492-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:52.552-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:52.552-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:52.591-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
PipelineManager 2017-04-25T13:04:52.833-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:50.001726005-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:04:52.977-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:04:52.992-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:04:53.167-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:53.212-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:04:53.509-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:04:53.509-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:04:53.509-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:04:53.509-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:53.509-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:53.598-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:53.598-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:53.610-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:04:53.794-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:53.853-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:04:53.918-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:04:53.940-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:04:54.324-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:04:54.324-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:04:54.324-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:04:54.324-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:54.324-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:54.425-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:54.425-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:54.445-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:04:54.496-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:04:54.513-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:04:54.781-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:54.800-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:04:55.009-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:04:55.009-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:04:55.009-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:04:55.009-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:55.009-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:55.090-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:55.090-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:55.115-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:04:55.206-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:55.224-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:04:55.489-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:55.511-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:04:55.645-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:04:55.645-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:04:55.645-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:04:55.645-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:55.645-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:55.725-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:55.725-05:00 [ERROR] batchUpdateDocs for vb 8 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:55.743-05:00 [INFO] Retrying update docs for vb 8 for the 1th time
CapiNozzle 2017-04-25T13:04:55.820-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:55.842-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:04:56.007-05:00 [ERROR] batchUpdateDocs for vb 8 failed with err receieved unexpected status code, 500, from update docs request for vb 8
.
CapiNozzle 2017-04-25T13:04:56.031-05:00 [INFO] Retrying update docs for vb 8 for the 2th time
CapiNozzle 2017-04-25T13:04:56.386-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:04:56.386-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:04:56.386-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:04:56.386-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:56.386-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:56.536-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:56.537-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:56.564-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:04:56.711-05:00 [ERROR] batchUpdateDocs for vb 8 failed with err receieved unexpected status code, 500, from update docs request for vb 8
.
CapiNozzle 2017-04-25T13:04:56.732-05:00 [INFO] Retrying update docs for vb 8 for the 3th time
CapiNozzle 2017-04-25T13:04:57.065-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:57.091-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:04:57.232-05:00 [ERROR] batchUpdateDocs for vb 8 failed with err receieved unexpected status code, 500, from update docs request for vb 8
.
CapiNozzle 2017-04-25T13:04:57.232-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 8 after 3 retries
CapiNozzle 2017-04-25T13:04:57.232-05:00 [ERROR] Raise error condition batch update docs failed for vb 8 after 3 retries
GenericSupervisor 2017-04-25T13:04:57.232-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:57.232-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:57.371-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:57.373-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:57.424-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:04:57.429-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:57.443-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:04:57.921-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:57.935-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:04:58.174-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:04:58.174-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:04:58.174-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:04:58.174-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:58.174-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:58.280-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:58.280-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:58.298-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:04:58.412-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:58.424-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:04:58.693-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:04:58.705-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:04:58.802-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:04:58.821-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:04:58.821-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:04:58.821-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:58.826-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:58.914-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:58.915-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:58.926-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
CapiNozzle 2017-04-25T13:04:59.140-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:04:59.152-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:04:59.322-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:04:59.333-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
GenericSupervisor 2017-04-25T13:04:59.344-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:04:59.344-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:04:59.344-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:04:59.609-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:04:59.612-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:04:59.612-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:04:59.612-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:04:59.612-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:04:59.612-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:04:59.687-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:04:59.687-05:00 [ERROR] batchUpdateDocs for vb 305 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:04:59.698-05:00 [INFO] Retrying update docs for vb 305 for the 1th time
CapiNozzle 2017-04-25T13:04:59.779-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:04:59.790-05:00 [INFO] Retrying update docs for vb 105 for the 3th time
PipelineManager 2017-04-25T13:05:00.002-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:05:00.002-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:05:00.002-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:05:00.002-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
PipelineRuntimeCtx 2017-04-25T13:05:00.002-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:05:00.002-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:05:00.002-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:05:00.002-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:00.002362031 -0500 CDT
GenericSupervisor 2017-04-25T13:05:00.002-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:05:00.002-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:05:00.002-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:05:00.002-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:05:00.002-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:05:00.002-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:05:00.002-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:05:00.002-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:05:00.002-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:05:00.002-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-248158752 has been stopped
GenericPipeline 2017-04-25T13:05:00.003-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-248158752 is stopped
PipelineManager 2017-04-25T13:05:00.003-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:05:00.003-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:05:00.003-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
PipelineManager 2017-04-25T13:05:00.003-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:04:50.001726005-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:05:00.003-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
GenericPipeline 2017-04-25T13:05:00.003-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
PipelineManager 2017-04-25T13:05:00.003-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:05:00.003-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-248158752
AsyncListener 2017-04-25T13:05:00.003-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:00.003-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:00.003-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:00.003-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:05:00.003-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:05:00.003-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:05:00.003-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:05:00.004-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:05:00.004-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:05:00.004-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:05:00.004-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:05:00.004-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:05:00.004-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:05:00.004-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:05:00.004-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:00.004301551 -0500 CDT
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:05:00.004-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:00.004-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:05:00.004-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:00.004594492 -0500 CDT
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:05:00.005-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:05:00.005-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:05:00.005-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.005-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:05:00.005-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:00.005-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:05:00.006-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:05:00.006-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:05:00.006-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:00.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:05:00.006-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:05:00.006-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:00.006-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:05:00.007-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:05:00.007-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:00.00744293 -0500 CDT
DcpNozzle 2017-04-25T13:05:00.007-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:05:00.007-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:00.007-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:05:00.008-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:05:00.008-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:00.008655739 -0500 CDT
DcpNozzle 2017-04-25T13:05:00.008-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:05:00.008-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:00.008-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:05:00 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:05:00 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:05:00 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:05:00.201-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:05:00.202-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:05:00.202-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:05:00.355-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:05:00.356-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:05:00.356-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:05:00.356-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:05:00.356-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:05:00.356-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:05:00.356-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:05:00.356-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:05:00.357-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
GenericSupervisor 2017-04-25T13:05:00.360-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:05:00.360-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
GenericSupervisor 2017-04-25T13:05:00.360-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:00.360-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:00.360-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:00.360-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:00.360-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:05:00.360-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
CheckpointManager 2017-04-25T13:05:00.360-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
CapiUtils 2017-04-25T13:05:00.466-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:05:00.466-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:05:00.466-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:05:00.470-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
StatisticsManager 2017-04-25T13:05:00.471-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:05:00.471-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
XDCRFactory 2017-04-25T13:05:00.471-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:05:00.471-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159 is constructed, start it
GenericPipeline 2017-04-25T13:05:00.472-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[0 81 73 104 166 164 105 133 75 32 65 117 165 93 101 107 146 9 31 162 42 64 72 135 100 108 22 19 20 76 152 1 66 79 124 138 59 113 137 145 61 142 151 109 122 132 46 70 74 125 13 39 60 28 77 119 121 15 127 24 136 169 16 52 95 158 18 26 83 154 63 89 147 116 156 49 94 102 170 17 84 111 139 143 30 161 126 144 157 35 48 110 25 56 149 14 99 103 148 23 27 163 40 114 115 12 47 50 2 5 51 98 11 90 153 36 97 128 118 131 160 62 78 7 87 112 155 53 67 68 85 92 69 86 120 129 41 82 96 88 8 29 45 58 141 150 123 3 140 167 106 6 10 44 57 71 33 4 34 54 134 21 43 55 91 130 37 80 159 168 38] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[178 202 307 190 305 186 207 244 323 204 224 183 173 233 254 332 223 243 248 324 206 225 259 230 272 282 301 316 213 239 242 264 284 335 179 315 205 228 267 328 261 274 174 208 221 306 192 194 327 246 249 265 279 187 199 203 278 201 214 222 251 176 220 262 276 287 294 297 236 280 322 309 334 231 258 288 320 256 277 308 182 281 188 191 302 321 241 253 314 326 338 189 193 293 181 197 209 226 235 292 300 333 180 216 195 198 329 296 218 240 212 234 312 171 266 286 290 319 217 232 330 247 252 270 283 299 311 331 229 289 303 211 215 237 268 313 339 196 238 310 317 298 200 210 250 255 260 263 184 269 285 337 219 291 295 325 175 227 273 275 304 318 185 257 336 172 177 271 245 340]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[11 90 153 36 97 128 118 131 160 62 78 7 87 112 155 53 67 68 85 92 69 86 120 129 41 82 96 88 8 29 45 58 141 150 123 3 140 167 106 6 10 44 57 71 33 4 34 54 134 21 43 55 91 130 37 80 159 168 38 0 81 73 104 166 164 105 133 75 32 65 117 165 93 101 107 146 9 31 162 42 64 72 135 100 108 22 19 20 76 152 1 66 79 124 138 59 113 137 145 61 142 151 109 122 132 46 70 74 125 13 39 60 28 77 119 121 15 127 24 136 169 16 52 95 158 18 26 83 154 63 89 147 116 156 49 94 102 170 17 84 111 139 143 30 161 126 144 157 35 48 110 25 56 149 14 99 103 148 23 27 163 40 114 115 12 47 50 2 5 51 98] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[240 212 234 312 171 266 286 290 319 217 232 330 270 283 247 252 299 311 331 229 289 303 237 268 211 215 313 339 196 238 310 317 298 200 210 250 255 260 263 337 184 269 285 219 291 295 325 175 227 273 275 318 304 185 257 336 271 172 177 245 340 178 202 307 190 305 186 207 244 323 204 224 183 173 233 254 332 324 223 243 248 259 206 225 316 230 272 282 301 213 242 264 239 284 335 179 315 328 205 228 267 261 274 306 174 208 221 327 192 194 246 249 265 279 278 187 199 203 222 251 201 214 262 276 176 220 287 294 297 236 280 322 309 334 231 258 288 320 256 308 277 182 281 188 191 302 321 241 253 314 326 338 293 189 193 226 235 181 197 209 292 300 333 180 216 195 198 329 296 218]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 filter_expression: active:true doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info replication_type:capi checkpoint_interval:1800 worker_batch_size:500 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:05:00.473-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159...
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:00.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
StatisticsManager 2017-04-25T13:05:00.484-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:05:00.502-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:05:00.502-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:05:00.503-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:05:00.514-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:05:00.532-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:05:00.532-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:05:00.532-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:05:00.532-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:05:00.532-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:05:00.532-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:05:00.532-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:05:00.532-05:00 [INFO] Checkpointing starts in 1802.289 sec
PipelineRuntimeCtx 2017-04-25T13:05:00.532-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:05:00.532-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:05:00.534-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:05:00.534-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:05:00.537-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:05:00.572-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:05:00.572-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:05:00.573-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:05:00.630-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:05:00.630-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:05:00.661-05:00 [INFO] Got the bucket metadata for ckmgr
ToplogyChangeDetector 2017-04-25T13:05:00.662-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:05:01.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:05:01.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:05:01.521-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:05:01.521-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:05:01.524-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:05:01.524-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:05:01.525-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:05:04.014-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:05:04.015-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:05:04.016-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:05:04.120-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:05:04.172-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:05:04.337-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:05:04.156-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:05:04.428-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:05:04.459-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:05:04.560-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:05:04 UPR_STREAMREQ for vb 0 successful
StatisticsManager 2017-04-25T13:05:04.824-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:05:04.893-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:05:04.893-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:05:04.893-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:05:04.970-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:05:04.974-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:05:04.976-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159 is started----------
PipelineManager 2017-04-25T13:05:04.976-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:05:04.976-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:05:04.976-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
DcpNozzle 2017-04-25T13:05:04.953-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
2017/04/25 13:05:05 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:05:05.046-05:00 [INFO] Send batch count=11
2017/04/25 13:05:05 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:05:05.161-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:05.173-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:05.186-05:00 [INFO] Send batch count=194
2017/04/25 13:05:05 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 4 successful
CapiNozzle 2017-04-25T13:05:05.345-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:05:05.421-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:05.421-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:05.433-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:05:05 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:05:05.682-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:05:05.702-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
StatisticsManager 2017-04-25T13:05:05.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:05:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 58 successful
CapiNozzle 2017-04-25T13:05:05.774-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:05:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 259 successful
CapiNozzle 2017-04-25T13:05:05.785-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:05:05.790-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:05:05.806-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:05:05.842-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:05:05.842-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:05:05.842-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:05:05.842-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:05:05.842-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:05:05 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 274 successful
PipelineManager 2017-04-25T13:05:05.892-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:05:05.893-05:00 [INFO] Send batch count=500
2017/04/25 13:05:05 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 268 successful
PipelineManager 2017-04-25T13:05:05.896-05:00 [INFO] err_list=[{"time":"2017-04-25T13:05:05.896748554-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:05:05.899-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:05:05 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:05:05 UPR_STREAMREQ for vb 288 successful
CapiNozzle 2017-04-25T13:05:06.050-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:06.058-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:05:06 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:05:06 UPR_STREAMREQ for vb 294 successful
StatisticsManager 2017-04-25T13:05:06.574-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:05:06.578-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
DcpNozzle 2017-04-25T13:05:06.406-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:05:06.616-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
StatisticsManager 2017-04-25T13:05:06.821-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159 {"Errors": "[{\"time\":\"2017-04-25T13:05:05.896748554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 115, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 86, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 1625, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 86, "set_filtered": 0, "set_received_from_dcp": 1625, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:05:06.780-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CheckpointManager 2017-04-25T13:05:07.421-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
CheckpointManager 2017-04-25T13:05:07.422-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:05:07.489-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:05:07 UPR_STREAMREQ for vb 195 successful
CapiNozzle 2017-04-25T13:05:07.790-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:07.975-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:05:07.975-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:05:08.122-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:05:07.975-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:05:08.289-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:05:08.434-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:05:08.528-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:08.696-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:08.707-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:05:08.804-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:05:08.815-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:05:08.900-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:08.921-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
ReplicationManager 2017-04-25T13:05:08.925-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:05:08 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:05:08 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:05:08 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:05:08 UPR_STREAMREQ for vb 199 successful
PipelineManager 2017-04-25T13:05:09.047-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:05:05.896748554-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:05:09.250-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:05:09.250-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:05:09.250-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:05:09.250-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:05:09.250-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:05:09.351-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:09.351-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:09.362-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:05:09.417-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:09.428-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:05:09.765-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:05:09.778-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:05:09.880-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:09.880-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:05:09.880-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:05:09.880-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:05:09.880-05:00 [INFO] Send batch count=239
CapiNozzle 2017-04-25T13:05:10.031-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:10.031-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:10.045-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:05:10.075-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:05:10.100-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:05:10.520-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:05:10.520-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:05:10.520-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:05:10.520-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:05:10.521-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:05:10.535-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:05:10.535-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:05:10.535-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:05:10.645-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:10.646-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:10.657-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:05:10.792-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:10.802-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:05:13.069-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
ToplogyChangeDetector 2017-04-25T13:05:12.774-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:05:13.660-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:05:13.627-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:15.163-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:05:15.702-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:05:15.715-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:05:15.905-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:05:15.905-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:05:15.905-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:05:15.905-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:05:15.905-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:05:15.911-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:05:15.911-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:05:15.911-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:05:15.911-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
PipelineRuntimeCtx 2017-04-25T13:05:15.911-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:05:15.911-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:05:15.911-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:05:15.915-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:15.911288374 -0500 CDT
GenericSupervisor 2017-04-25T13:05:15.915-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:05:15.915-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:05:15.915-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:05:15.915-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:05:15.915-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:05:15.915-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:05:15.915-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:05:15.915-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:05:15.915-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:05:15.915-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-356632159 has been stopped
GenericPipeline 2017-04-25T13:05:15.916-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-356632159 is stopped
PipelineManager 2017-04-25T13:05:15.916-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:05:15.916-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:05:05.896748554-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:05:15.916-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:05:15.916-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
GenericPipeline 2017-04-25T13:05:15.916-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
GenericPipeline 2017-04-25T13:05:15.916-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
GenericPipeline 2017-04-25T13:05:15.916-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-356632159
AsyncListener 2017-04-25T13:05:15.916-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:05:15.916-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:05:15.918-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:05:15.918-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:05:15.918-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:05:15.918-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:05:15.918-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:05:15.919-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.918-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:05:15.919-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:05:15.919-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:05:15.919-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:05:15.919-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:05:15.919-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:05:15.919-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:15.91904616 -0500 CDT
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:05:15.919-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:05:15.919-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:15.919457003 -0500 CDT
CapiNozzle 2017-04-25T13:05:15.919-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:15.919-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:05:15.920-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:05:15.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:05:15.920-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:05:15.921-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:05:15.921-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:15.921-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:05:15.921-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:05:15.921-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:15.921831077 -0500 CDT
DcpNozzle 2017-04-25T13:05:15.921-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:05:15.921-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:15.921-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:05:15.923-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:05:15.923-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:05:15.923168863 -0500 CDT
DcpNozzle 2017-04-25T13:05:15.923-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:05:15.923-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:15.923-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:05:15 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:05:15 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:05:15 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:05:15.994-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:05:15.994-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:05:15.994-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:05:15.994-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:05:16.284-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:05:16.285-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:05:16.285-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:05:16.336-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:05:16.337-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:05:16.337-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:05:16.337-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:05:16.337-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:05:16.337-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:05:16.337-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:05:16.338-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:05:16.338-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
GenericSupervisor 2017-04-25T13:05:16.341-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:05:16.341-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
GenericSupervisor 2017-04-25T13:05:16.341-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:16.341-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:16.341-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:16.341-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:16.341-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:05:16.341-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
CheckpointManager 2017-04-25T13:05:16.341-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
CapiUtils 2017-04-25T13:05:16.375-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:05:16.375-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:05:16.375-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:05:16.376-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
StatisticsManager 2017-04-25T13:05:16.377-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:05:16.377-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
XDCRFactory 2017-04-25T13:05:16.377-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:05:16.377-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 is constructed, start it
GenericPipeline 2017-04-25T13:05:16.378-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[5 114 132 109 112 140 141 143 147 100 120 105 168 30 53 14 102 32 111 3 27 59 121 158 13 46 167 90 103 145 12 76 138 170 33 81 99 118 163 166 97 129 107 86 89 31 160 44 122 127 155 29 62 2 34 153 157 71 82 159 94 117 20 38 101 133 37 70 4 52 57 73 91 95 162 164 39 43 58 61 124 128 1 41 93 74 83 146 151 60 96 8 9 22 78 139 49 75 136 10 67 56 65 135 50 55 16 23 169 6 110 130 131 144 17 63 24 40 7 18 116 148 149 15 72 69 106 108 126 36 123 77 80 150 0 51 85 154 66 45 54 87 88 125 134 142 152 25 26 137 156 21 42 104 119 161 19 28 48 165 92 84 68 98 35 47 115 79 113 11 64] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[245 262 277 205 229 241 331 319 233 212 254 284 281 192 250 291 183 286 312 327 181 239 273 279 234 295 296 187 191 307 236 251 276 338 207 216 269 271 275 311 171 226 305 321 292 340 223 225 227 252 189 310 320 306 217 300 235 298 193 218 266 270 228 297 180 199 230 249 202 248 206 263 285 334 257 267 332 242 289 185 203 179 178 177 219 313 336 304 316 172 246 190 211 290 309 318 329 204 215 220 244 280 224 232 247 268 314 238 337 176 256 278 175 188 272 294 322 174 308 209 260 335 259 317 186 324 323 182 214 253 265 264 210 243 221 240 173 184 196 198 213 255 315 293 299 201 237 274 283 288 197 258 194 222 328 200 195 325 330 287 208 282 301 302 339 303 326 261 231 333]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[260 335 209 259 317 186 324 182 323 214 253 265 264 210 243 173 184 221 240 255 315 196 198 213 201 293 299 237 274 283 288 194 197 258 200 222 328 195 325 330 287 282 301 302 208 339 261 303 326 333 231 262 277 245 205 229 331 241 319 233 212 254 284 281 192 250 291 286 312 327 183 279 181 239 273 187 234 295 296 191 307 276 338 236 251 207 216 269 271 311 275 321 171 226 305 340 292 252 189 223 225 227 306 310 320 217 300 298 235 270 193 218 266 180 199 228 297 230 202 248 249 206 263 285 334 332 257 267 185 203 242 289 179 178 177 219 313 336 172 246 304 316 309 318 329 190 211 290 244 280 204 215 220 247 268 314 224 232 238 337 176 256 278 272 294 322 175 188 174 308] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[7 18 24 40 15 72 116 148 149 69 106 108 36 123 126 0 51 77 80 150 85 154 45 54 66 134 142 152 25 26 87 88 125 137 156 21 161 19 28 42 104 119 48 165 92 84 35 47 68 98 115 11 64 79 113 5 109 112 114 132 147 100 120 140 141 143 30 53 105 168 14 102 3 27 32 111 13 46 59 121 158 167 12 76 90 103 145 33 81 138 170 99 118 163 97 129 166 86 89 107 31 160 44 122 29 62 127 155 2 34 71 82 153 157 159 94 20 38 117 37 70 101 133 4 52 95 162 164 39 43 57 73 91 128 1 41 58 61 124 74 83 93 146 151 60 96 8 9 22 78 139 49 75 10 67 136 50 55 56 65 135 16 23 6 110 169 17 63 130 131 144]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: active:true worker_batch_size:500 failure_restart_interval:10 target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:05:16.378-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336...
AsyncListener 2017-04-25T13:05:16.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:16.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:16.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:16.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:16.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:05:16.388-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
CheckpointManager 2017-04-25T13:05:16.388-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:05:16.388-05:00 [INFO] Checkpointing starts in 1802.131 sec
PipelineRuntimeCtx 2017-04-25T13:05:16.388-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:05:16.388-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:05:16.388-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:05:16.415-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:05:16.415-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:05:16.415-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:05:16.416-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:05:16.416-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:05:16.416-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:05:16.416-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:05:16.416-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:05:16.417-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:05:16.417-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:05:16.417-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:05:16.417-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:05:16.430-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:05:16.459-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:05:16.465-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:05:16.465-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:05:16.485-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:05:16.500-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:05:16.500-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:05:16.522-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:05:16.555-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:05:16.605-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:05:16.605-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:05:16.605-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:05:16.610-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:05:16.610-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:05:16.611-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:05:16.642-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:05:16.642-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:05:16.642-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:05:16.672-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:05:16.672-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:05:16.809-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:05:16.809-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:05:16.809-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
CheckpointManager 2017-04-25T13:05:16.815-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:05:16.824-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:05:16.824-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:05:16.824-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:05:16.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:05:16.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:05:16.834-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:05:16.835-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 is started----------
PipelineManager 2017-04-25T13:05:16.835-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:05:16.835-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:05:16.835-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:05:17 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:05:17 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:05:18 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:05:19 UPR_STREAMREQ for vb 41 successful
StatisticsManager 2017-04-25T13:05:19.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:05:20 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 250 successful
StatisticsManager 2017-04-25T13:05:20.057-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:05:20 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 275 successful
StatisticsManager 2017-04-25T13:05:20.631-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:05:20 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:05:20 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:05:21 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:05:21 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:05:21 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:05:21 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:05:21 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:05:22 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:05:23 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:05:23 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:05:23 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:05:24 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:05:23 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:05:24 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:05:24 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:05:25 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:05:26 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:05:29 UPR_STREAMREQ for vb 190 successful
GenericSupervisor 2017-04-25T13:05:29.147-05:00 [INFO] About to call function
2017/04/25 13:05:29 UPR_STREAMREQ for vb 90 successful
StatisticsManager 2017-04-25T13:05:29.318-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:05:29.192-05:00 [INFO] validateSourceTopology...
2017/04/25 13:05:32 UPR_STREAMREQ for vb 293 successful
GenericSupervisor 2017-04-25T13:05:30.390-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:05:37.167-05:00 [INFO] ****************************
2017/04/25 13:05:39 UPR_STREAMREQ for vb 191 successful
GenericSupervisor 2017-04-25T13:05:38.857-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:05:40.225-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
2017/04/25 13:05:40 UPR_STREAMREQ for vb 294 successful
GenericSupervisor 2017-04-25T13:05:41.237-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:05:40.648-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:05:40.186-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:05:41.321-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:05:41.370-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:05:05.896748554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 1, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [98 97 92 93 91 95 96 94 99]
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [296 197 192 199 299 194 298 193 196 295 198 195 297]
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
2017/04/25 13:05:41 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:05:41 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:05:41 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:05:42 UPR_STREAMREQ for vb 196 successful
ToplogyChangeDetector 2017-04-25T13:05:42.966-05:00 [INFO] validateSourceTopology...
PipelineManager 2017-04-25T13:05:42.978-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
GenericSupervisor 2017-04-25T13:05:42.980-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:05:42.980-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:05:42.980-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:05:43 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 197 successful
CheckpointManager 2017-04-25T13:05:43.027-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
CheckpointManager 2017-04-25T13:05:43.027-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:05:43.066-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:05:43 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 198 successful
DcpNozzle 2017-04-25T13:05:43.118-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:05:43 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:05:43 UPR_STREAMREQ for vb 99 successful
StatisticsManager 2017-04-25T13:05:43.149-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:05:43.274-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:05:44.960-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
StatisticsManager 2017-04-25T13:05:46.600-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:05:46.645-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:05:46.645-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:05:51.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:05:50.690-05:00 [INFO] Executing 0x681fb0 timed out
CapiNozzle 2017-04-25T13:05:55.690-05:00 [INFO] Send batch count=382
GenericSupervisor 2017-04-25T13:05:51.958-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:05:55.905-05:00 [INFO] Send batch count=1
GenericSupervisor 2017-04-25T13:05:56.203-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:05:56.527-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:05:05.896748554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 1, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 250, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 730, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 871934, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 3559, "expiry_received_from_dcp": 0, "set_received_from_dcp": 3559}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 201, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 3901, "expiry_received_from_dcp": 0, "set_received_from_dcp": 3901}}

StatisticsManager 2017-04-25T13:05:56.530-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 3901 items, sent 1 items
StatisticsManager 2017-04-25T13:05:56.530-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 3559 items, sent 382 items
StatisticsManager 2017-04-25T13:05:56.537-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:3559]
StatisticsManager 2017-04-25T13:05:56.537-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 3559 items, sent 3559 items.
StatisticsManager 2017-04-25T13:05:56.537-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:3901]
StatisticsManager 2017-04-25T13:05:56.537-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 3901 items, sent 3901 items.
StatisticsManager 2017-04-25T13:05:56.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.541-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:05:05.896748554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 1, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 250, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 730, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 871934, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 3559, "expiry_received_from_dcp": 0, "set_received_from_dcp": 3559}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 201, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 3901, "expiry_received_from_dcp": 0, "set_received_from_dcp": 3901}}

StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 3559 items, sent 382 items
StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 3901 items, sent 1 items
StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:3559]
StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 3559 items, sent 3559 items.
StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:3901]
StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 3901 items, sent 3901 items.
StatisticsManager 2017-04-25T13:05:56.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
GenericSupervisor 2017-04-25T13:05:56.548-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:05:56.597-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:05:56.598-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
ToplogyChangeDetector 2017-04-25T13:05:56.675-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:05:56.700-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:05:56.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:05:56.942-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:56.982-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:56.942-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:05:56.993-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:05:57.104-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:05:57.127-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
StatisticsManager 2017-04-25T13:05:57.449-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:05:57.469-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:05:57.484-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
GenericSupervisor 2017-04-25T13:05:57.532-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:05:57.532-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:05:57.532-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:05:57.715-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:05:57.748-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:05:57.821-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:05:57.868-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
PipelineManager 2017-04-25T13:05:57.950-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:05:58.210-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:05:58.226-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:05:58.343-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:05:58.343-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:05:58.343-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:05:58.347-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries]
ReplicationManager 2017-04-25T13:05:58.350-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries]
StatisticsManager 2017-04-25T13:05:58.430-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:05:58.431-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 184, "dcp_dispatch_time": 28, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 505, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 22103, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 505, "set_filtered": 0, "set_received_from_dcp": 22103, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

PipelineManager 2017-04-25T13:05:58.631-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:05:58.635-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:05:59.083-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
PipelineManager 2017-04-25T13:05:58.640-05:00 [INFO] err_list=[{"time":"2017-04-25T13:05:58.640283469-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries"}]
CapiNozzle 2017-04-25T13:06:01.910-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:06:06.478-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:06:06.834-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:06:06.715-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
ToplogyChangeDetector 2017-04-25T13:06:06.823-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:06:06.840-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:06:07.453-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:06:06.840-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:06:07.726-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:06:10.585-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:06:12.360-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:06:12.917-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:06:12.920-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:06:13.147-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:06:13.162-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:06:13.093-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:06:13.200-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:06:13.254-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:06:13.272-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
ReplicationManager 2017-04-25T13:06:13.650-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:06:14.792-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:06:16.133-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:06:16.091-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
PipelineManager 2017-04-25T13:06:19.284-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries 
CapiNozzle 2017-04-25T13:06:20.916-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
PipelineManager 2017-04-25T13:06:20.866-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:06:20.964-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:06:20.967-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
PipelineRuntimeCtx 2017-04-25T13:06:20.970-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:06:21.060-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:06:21.068-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:06:21.094-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:06:21.219-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:06:21.080257807 -0500 CDT
GenericSupervisor 2017-04-25T13:06:21.787-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:06:21.796-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:06:21.848-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:06:21.848-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CheckpointManager 2017-04-25T13:06:21.840-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:06:21.969-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:06:22.066-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:06:23.921-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:06:26.821-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 has been stopped
GenericSupervisor 2017-04-25T13:06:30.135-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:06:30.613-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:06:32.057-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:06:32.031-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:06:32.703-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:06:35.048-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
GenericPipeline 2017-04-25T13:06:35.067-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
GenericPipeline 2017-04-25T13:06:35.077-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
AsyncListener 2017-04-25T13:06:35.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:06:35.098-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:06:35.099-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:06:35.099-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:06:35.101-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:06:35.101-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:06:35.102-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-337943336 is stopped
PipelineManager 2017-04-25T13:06:35.102-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:06:35.116-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:06:35.117-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:06:35.123-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:06:35.123-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:06:35.127-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:06:35.128-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:06:35.12394022 -0500 CDT
CapiNozzle 2017-04-25T13:06:35.144-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:06:35.144-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
GenericPipeline 2017-04-25T13:06:35.147-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-337943336
CapiNozzle 2017-04-25T13:06:35.148-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:06:35.149-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.155-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:06:35.155-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:06:35.156-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.157-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:06:35.157-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:06:35.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:06:35.159-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:06:35.168-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.168-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:06:35.169-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericPipeline 2017-04-25T13:06:35.169-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:06:35.170-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:06:35.103-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:06:35.170-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:06:35.171-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:06:35.078-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
2017/04/25 13:06:35 Stream Closed for vb 0 StreamEnd simulated
PipelineManager 2017-04-25T13:06:35.104-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:05:58.640283469-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:06:35.235-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:06:35.237-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:06:35.237-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:06:35.237-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:06:35.237-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:06:35.237-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:06:35.237634299 -0500 CDT
CapiNozzle 2017-04-25T13:06:35.237-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:06:35.237-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:06:35.237-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:06:35.258-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:06:35 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:06:35 Stream Closed for vb 2 StreamEnd simulated
DcpNozzle 2017-04-25T13:06:35.258-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:06:35.258200235 -0500 CDT
DcpNozzle 2017-04-25T13:06:35.258-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:06:35.258-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:06:35.258-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:06:35.293-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:06:35.306-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:06:35.306-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:06:35.306-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:06:35.322-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:06:35.322-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:06:35.322603455 -0500 CDT
DcpNozzle 2017-04-25T13:06:35.322-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:06:35.322-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:06:35.322-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
PipelineManager 2017-04-25T13:06:35.614-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:05:58.640283469-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:06:35.635-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
PipelineManager 2017-04-25T13:06:35.643-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:05:58.640283469-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:06:35.644-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:06:35.644-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:06:35.684-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:06:35.689-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:06:35.693-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:06:35.693-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:06:35.693-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:06:35.693-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:06:35.693-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:06:35.694-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:06:35.700-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
GenericSupervisor 2017-04-25T13:06:35.705-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:06:35.706-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
GenericSupervisor 2017-04-25T13:06:35.706-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:06:35.706-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:06:35.706-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:06:35.706-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:06:35.706-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:06:35.710-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
CheckpointManager 2017-04-25T13:06:35.710-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
CapiUtils 2017-04-25T13:06:35.741-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:06:35.741-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:06:35.745-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:06:35.759-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
StatisticsManager 2017-04-25T13:06:35.772-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:06:35.774-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
XDCRFactory 2017-04-25T13:06:35.774-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:06:35.775-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 is constructed, start it
GenericPipeline 2017-04-25T13:06:35.777-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[4 14 125 65 96 122 45 133 121 124 138 26 90 93 98 149 5 77 151 99 114 144 158 21 57 73 22 16 156 167 71 76 152 132 13 115 84 157 28 56 82 129 168 81 110 29 35 41 111 20 102 154 153 52 116 137 17 83 131 170 44 54 74 143 24 38 127 140 25 47 126 30 92 49 100 113 95 164 0 60 66 108 145 148 169 58 85 86 106 7 46 62 42 68 128 134 33 59 120 78 139 6 89 3 69 103 142 39 64 135 1 161 166 51 55 70 2 119 50 53 80 72 37 94 141 31 87 123 10 79 36 8 12 19 101 155 11 63 130 27 15 40 34 105 109 18 48 9 88 112 159 146 160 136 32 91 118 117 147 162 67 104 107 61 165 23 43 163 75 97 150] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[192 242 335 181 197 235 288 302 306 175 336 264 283 177 227 255 259 263 180 200 226 256 277 280 239 190 233 248 294 312 246 189 321 323 338 225 215 196 314 214 178 316 289 301 317 176 204 195 253 308 273 278 293 191 265 292 232 287 179 184 217 329 285 318 257 270 207 319 182 333 172 313 328 222 223 238 254 219 250 337 247 284 298 311 216 212 322 194 274 315 300 251 271 244 174 211 218 220 272 282 330 245 275 286 309 310 185 228 229 208 230 334 249 281 304 231 243 199 260 290 198 213 320 202 224 307 236 237 299 186 206 188 203 258 187 173 279 326 234 193 210 276 325 332 201 209 291 303 261 339 205 266 297 267 269 296 340 171 241 295 305 221 240 262 324 327 331 268 183 252]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[106 7 46 62 42 68 128 33 59 120 134 78 139 6 89 3 69 103 39 64 135 142 1 161 166 51 55 70 2 119 50 53 80 72 37 94 31 87 123 141 10 79 36 8 12 19 101 155 11 63 130 27 15 40 34 105 109 18 48 9 88 112 159 146 160 32 91 118 136 67 104 107 117 147 162 61 165 23 43 75 97 150 163 4 14 65 96 122 125 45 133 26 90 93 121 124 138 98 149 5 77 151 21 57 73 99 114 144 158 22 16 156 71 76 152 167 132 13 115 28 56 82 84 157 129 168 29 35 41 81 110 111 20 102 154 52 116 137 153 17 44 54 74 83 131 170 24 38 127 143 25 47 126 140 30 92 49 100 113 0 60 66 95 164 58 85 86 108 145 148 169] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[207 319 182 172 313 328 333 222 223 238 254 219 250 337 311 247 284 298 212 216 194 322 274 315 300 244 251 271 272 174 211 218 220 282 330 310 185 228 245 275 286 309 208 229 230 249 281 304 334 199 231 243 260 198 290 213 320 202 224 307 186 206 236 237 299 187 188 203 258 173 279 326 193 234 201 209 210 276 325 332 261 291 303 205 266 297 339 171 241 267 269 296 340 221 295 305 262 240 324 327 268 331 183 252 181 192 242 335 306 175 197 235 288 302 336 177 264 283 227 255 259 263 180 200 226 256 277 239 280 190 233 246 248 294 312 323 189 321 338 215 225 196 314 214 178 316 176 204 289 301 317 253 195 273 278 293 308 191 265 292 232 179 287 184 217 329 285 318 257 270]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 stats_interval:1000 replication_type:capi active:true worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:06:35.780-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687...
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:06:35.784-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
StatisticsManager 2017-04-25T13:06:35.785-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:06:35.796-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:06:35.796-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:06:35.797-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:06:35.797-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:06:35.798-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:06:35.798-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:06:35.798-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:06:35.799-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:06:35.799-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:06:35.800-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:06:35.803-05:00 [INFO] Checkpointing starts in 1800.659 sec
PipelineRuntimeCtx 2017-04-25T13:06:35.803-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:06:35.804-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:06:35.812-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:06:35.813-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:06:35.814-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:06:35.820-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:06:35.835-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:06:35.853-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:06:35.853-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:06:35.853-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:06:35.874-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:06:35.874-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:06:35.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:06:35.928-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:06:35.963-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:06:35.963-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:06:35.963-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:06:35.969-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:06:35.970-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:06:35.971-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:06:35.986-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:06:35.986-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:06:35.986-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:06:36.011-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:06:36.011-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:06:36.158-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:06:36.172-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:06:36.172-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:06:36.172-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:06:36.195-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:06:36.195-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:06:36.195-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:06:36.196-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:06:36.196-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:06:36.196-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:06:36.196-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 is started----------
PipelineManager 2017-04-25T13:06:36.196-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:06:36.196-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:06:36.197-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:06:36 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 205 successful
CapiNozzle 2017-04-25T13:06:36.488-05:00 [INFO] Send batch count=163
2017/04/25 13:06:36 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 208 successful
CapiNozzle 2017-04-25T13:06:36.516-05:00 [INFO] Send batch count=156
CapiNozzle 2017-04-25T13:06:36.549-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:06:36.554-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:06:36.571-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
2017/04/25 13:06:36 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 137 successful
CapiNozzle 2017-04-25T13:06:36.872-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:06:36 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:06:36.908-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:06:36 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:06:36 UPR_STREAMREQ for vb 243 successful
CapiNozzle 2017-04-25T13:06:36.965-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:06:37.264-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
2017/04/25 13:06:39 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:06:39 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:06:39 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:06:39 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:06:39 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:06:39 UPR_STREAMREQ for vb 244 successful
StatisticsManager 2017-04-25T13:06:40.087-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
GenericSupervisor 2017-04-25T13:07:26.072-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:07:26.044-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:07:26.094-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
StatisticsManager 2017-04-25T13:07:26.271-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:05:58.640283469-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 5 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 20, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 138, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 8459, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 8459, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 138, "set_filtered": 0, "set_received_from_dcp": 8459, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 21}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 66, "docs_opt_repd": 0, "docs_rep_queue": 6717, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 66, "size_rep_queue": 9104652, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 128, "max": 16, "mean": 15.6171875, "min": 15}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 72, "docs_opt_repd": 0, "docs_rep_queue": 7068, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 72, "size_rep_queue": 9183396, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 132, "max": 27, "mean": 27, "min": 27}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 3, "dcp_dispatch_time": {"count": 3821, "max": 3515, "mean": 14.644, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 6902, "expiry_received_from_dcp": 0, "set_received_from_dcp": 6902}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 1, "dcp_dispatch_time": {"count": 4638, "max": 9671, "mean": 24.934, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 7248, "expiry_received_from_dcp": 0, "set_received_from_dcp": 7248}}

StatisticsManager 2017-04-25T13:07:26.271-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 6902 items, sent 156 items
StatisticsManager 2017-04-25T13:07:26.271-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 7248 items, sent 163 items
GenericSupervisor 2017-04-25T13:07:26.271-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:07:26.290-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:26.294-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:26.338-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:26.354-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:07:26.356-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:6929]
StatisticsManager 2017-04-25T13:07:26.357-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 6929 items, sent 6929 items. streams inactive: [85 84 96 158 164 80 86 71 74 55 64 76 89 63 88 95 155 170 57 62 70 77 92 157 59 91 168 58 66 72 93 159 61 162 167 75 81 97 83 99 165 82 94 65 73 56 68 90 98 169 161 163 53 67 78 160 54 69 87 166 60 156 79]
StatisticsManager 2017-04-25T13:07:26.357-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:7278]
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 7278 items, sent 7278 items. streams inactive: [253 289 279 284 291 294 192 257 267 292 295 179 195 260 270 288 194 280 256 252 262 188 197 254 268 299 193 264 272 276 181 180 266 277 297 176 246 263 283 198 251 281 187 189 293 172 177 186 178 248 286 296 173 185 287 290 175 247 258 259 278 196 271 261 285 199 275 282 298 249 269 273 184 190 174 250 274 191 265 171 183 255 182]
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =27 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =27 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:07:26.358-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
2017/04/25 13:07:26 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:07:26 UPR_STREAMREQ for vb 53 successful
CapiNozzle 2017-04-25T13:07:26.493-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:07:26.550-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:07:26 UPR_STREAMREQ for vb 246 successful
GenericSupervisor 2017-04-25T13:07:27.106-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:07:27.107-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:07:27.107-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:07:27.147-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:07:27 UPR_STREAMREQ for vb 247 successful
ToplogyChangeDetector 2017-04-25T13:07:27.416-05:00 [INFO] validateSourceTopology...
2017/04/25 13:07:27 UPR_STREAMREQ for vb 248 successful
StatisticsManager 2017-04-25T13:07:27.504-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:07:27 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:07:27 UPR_STREAMREQ for vb 257 successful
PipelineManager 2017-04-25T13:07:27.915-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:07:33 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 164 successful
CapiNozzle 2017-04-25T13:07:33.124-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:07:33 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 65 successful
CapiNozzle 2017-04-25T13:07:33.765-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:07:33.767-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:07:33.767-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
2017/04/25 13:07:33 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:07:33 UPR_STREAMREQ for vb 168 successful
CapiNozzle 2017-04-25T13:07:34.035-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
GenericSupervisor 2017-04-25T13:07:33.770-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]
ReplicationManager 2017-04-25T13:07:34.047-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]
2017/04/25 13:07:34 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:07:34 UPR_STREAMREQ for vb 68 successful
PipelineManager 2017-04-25T13:07:34.530-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:07:34.532-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:07:34.571-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
PipelineManager 2017-04-25T13:07:34.554-05:00 [INFO] err_list=[{"time":"2017-04-25T13:07:34.554986841-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries"}]
PipelineManager 2017-04-25T13:07:34.714-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:07:34.717-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:07:34.719-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 {"Errors": "[{\"time\":\"2017-04-25T13:07:34.554986841-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 4, "dcp_dispatch_time": 498, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 138, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 17012, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 138, "set_filtered": 0, "set_received_from_dcp": 17012, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]", "Status": "Pending"}

2017/04/25 13:07:35 UPR_STREAMREQ for vb 169 successful
GenericSupervisor 2017-04-25T13:07:36.348-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:07:36.336-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:07:36.356-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:07:36.536-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:07:37 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:07:37 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:07:37.955-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:38.272-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:07:38 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:07:38 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:07:38 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:07:38 UPR_STREAMREQ for vb 73 successful
CapiNozzle 2017-04-25T13:07:39.051-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:07:39.155-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:07:39.155-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:07:39.155-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:07:39.155-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:39.165-05:00 [INFO] Send batch count=500
2017/04/25 13:07:39 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 273 successful
ReplicationManager 2017-04-25T13:07:39.431-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:07:39 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 81 successful
CapiNozzle 2017-04-25T13:07:39.687-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:39.687-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:39.703-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:07:39 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 277 successful
CapiNozzle 2017-04-25T13:07:39.871-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:07:39 UPR_STREAMREQ for vb 188 successful
CapiNozzle 2017-04-25T13:07:39.893-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
2017/04/25 13:07:39 UPR_STREAMREQ for vb 85 successful
PipelineManager 2017-04-25T13:07:39.948-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:07:34.554986841-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]}
]
2017/04/25 13:07:39 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:07:39 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 189 successful
CapiNozzle 2017-04-25T13:07:40.032-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:07:40.056-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:07:40.134-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:07:40 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 280 successful
CapiNozzle 2017-04-25T13:07:40.168-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
2017/04/25 13:07:40 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 195 successful
CapiNozzle 2017-04-25T13:07:40.482-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:07:40 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 90 successful
CapiNozzle 2017-04-25T13:07:40.502-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
2017/04/25 13:07:40 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 287 successful
CapiNozzle 2017-04-25T13:07:40.641-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:07:40.641-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:07:40.641-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:07:40.641-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:40.641-05:00 [INFO] Send batch count=500
2017/04/25 13:07:40 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 97 successful
CapiNozzle 2017-04-25T13:07:40.769-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:40.769-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:07:40 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 98 successful
CapiNozzle 2017-04-25T13:07:40.793-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:07:40 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 293 successful
DcpNozzle 2017-04-25T13:07:40.880-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:07:40 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:07:40 UPR_STREAMREQ for vb 296 successful
CheckpointManager 2017-04-25T13:07:40.934-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
CheckpointManager 2017-04-25T13:07:40.934-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:07:41 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:07:41 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:07:41.007-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:07:41.007-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:07:41.007-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:07:41.007-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:41.007-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:41.101-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:41.101-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
DcpNozzle 2017-04-25T13:07:41.111-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:07:41.129-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:07:41.179-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:07:41.206-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:07:41.776-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:07:41.793-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:07:42.114-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:07:42.138-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:07:42.424-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:07:42.444-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:07:42.792-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:07:42.792-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:07:42.792-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:07:42.792-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:42.792-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:42.889-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:42.889-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:42.918-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:07:43.210-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:07:43.210-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:07:43.210-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:07:43.210-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:43.210-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:43.378-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:43.378-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:43.393-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:07:43.512-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:07:43.529-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:07:43.854-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:07:43.880-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:07:44.088-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:07:44.114-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:07:44.266-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:07:44.281-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:07:44.436-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:07:44.436-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:07:44.436-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:07:44.436-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:44.436-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:44.528-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:44.528-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:44.539-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:07:44.680-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:07:44.681-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:07:44.681-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:07:44.681-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:44.681-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:07:44.717-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries 
PipelineManager 2017-04-25T13:07:44.717-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:07:44.717-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:07:44.719-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
PipelineRuntimeCtx 2017-04-25T13:07:44.722-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:07:44.727-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:07:44.727-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:07:44.737-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:44.727304625 -0500 CDT
GenericSupervisor 2017-04-25T13:07:44.742-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:07:44.742-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:07:44.743-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:07:44.743-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:07:44.743-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:07:44.743-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:07:44.744-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:07:44.744-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:07:44.744-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:07:44.744-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 has been stopped
GenericPipeline 2017-04-25T13:07:44.746-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-693712687 is stopped
PipelineManager 2017-04-25T13:07:44.746-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:07:44.747-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:07:34.554986841-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:07:44.749-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:07:44.759-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:44.759-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericPipeline 2017-04-25T13:07:44.760-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
GenericPipeline 2017-04-25T13:07:44.760-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
GenericPipeline 2017-04-25T13:07:44.760-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
GenericPipeline 2017-04-25T13:07:44.760-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-693712687
AsyncListener 2017-04-25T13:07:44.762-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:44.762-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.765-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:07:44.765-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:07:44.766-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:44.767-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:07:44.767-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:07:44.769-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:07:44.768-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:07:44.769-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.769-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:44.769-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:07:44.769-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:07:44.770-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:07:44.770-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:07:44.770-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:07:44.770-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:07:44.770-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:07:44.770-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:07:44.770-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:07:44.770-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:44.770745897 -0500 CDT
DcpNozzle 2017-04-25T13:07:44.771-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:07:44.771-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:07:44.771-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:07:44.771-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.771-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:07:44.771-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:07:44.771-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:44.771-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:07:44.774-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:07:44.774-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:07:44.774-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:44.774089357 -0500 CDT
DcpNozzle 2017-04-25T13:07:44.774-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:44.774100757 -0500 CDT
DcpNozzle 2017-04-25T13:07:44.774-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:07:44.774-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:07:44.774-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:44.774-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:07:44.774-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:44.774-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:07:44 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:07:44 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:07:44.774-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:44.769756358 -0500 CDT
CapiNozzle 2017-04-25T13:07:44.774-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:07:44.774-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.774-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:44.774-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.774-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:44.774-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:44.774-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:07:44.776-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:07:44 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:07:44.799-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:07:44.802-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:07:44.802-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:44.802-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:07:44.891-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:07:44.894-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:07:44.894-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:07:44.932-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:07:44.935-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:07:44.938-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:07:44.938-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:07:44.938-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:07:44.938-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:07:44.938-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:07:44.940-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:07:44.942-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
GenericSupervisor 2017-04-25T13:07:44.945-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:07:44.945-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
GenericSupervisor 2017-04-25T13:07:44.945-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:44.945-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:44.945-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:44.945-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:44.945-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:07:44.958-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
CheckpointManager 2017-04-25T13:07:44.958-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
CapiUtils 2017-04-25T13:07:44.984-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:07:44.984-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:07:44.984-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:07:44.986-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
StatisticsManager 2017-04-25T13:07:44.991-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:07:44.992-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
XDCRFactory 2017-04-25T13:07:44.992-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:07:44.992-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656 is constructed, start it
GenericPipeline 2017-04-25T13:07:44.993-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[302 183 197 203 286 338 182 184 316 252 260 263 215 280 284 289 328 238 248 309 334 201 243 287 326 333 339 217 240 271 272 308 249 269 223 264 291 206 298 225 292 195 189 265 329 230 239 246 171 270 301 279 323 254 261 331 185 209 256 274 322 204 221 224 324 337 315 303 245 294 310 179 193 226 321 191 312 332 267 200 218 222 174 177 278 325 235 251 186 187 180 297 173 175 192 229 266 290 194 207 314 318 205 281 268 276 213 214 335 242 283 306 219 247 317 330 216 293 181 228 282 253 273 285 305 231 237 196 296 220 255 190 236 299 176 241 199 244 277 340 258 211 212 313 232 233 288 198 227 250 311 336 320 210 234 257 275 304 172 319 327 208 307 188 259 295 300 178 202 262] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[6 80 44 118 145 51 124 27 38 39 131 136 163 55 59 81 101 153 53 68 157 25 26 165 22 0 78 147 42 49 84 97 144 40 76 23 89 113 160 110 96 127 151 33 46 56 155 2 17 28 114 141 100 134 83 21 86 87 98 112 31 108 4 65 149 154 47 67 132 103 109 148 37 79 85 122 133 12 32 45 64 111 7 150 152 77 91 143 11 63 170 69 107 121 146 102 105 159 8 36 130 71 164 54 61 104 15 156 57 93 116 120 94 140 161 10 16 162 9 125 3 35 62 70 123 18 73 41 66 169 168 138 158 128 137 43 1 19 106 139 5 24 95 126 142 30 50 60 135 13 29 34 90 129 115 20 48 75 167 72 82 92 58 74 119 166 52 99 88 117 14]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[94 140 161 10 16 162 9 3 35 62 70 123 125 18 73 41 66 169 168 138 158 128 137 43 1 19 106 139 5 24 95 126 142 30 50 60 13 29 34 90 129 135 115 20 48 75 167 72 82 92 58 74 119 166 52 99 88 117 14 6 80 44 118 145 51 124 27 38 39 131 136 55 59 81 101 153 163 53 68 157 25 26 165 22 0 78 147 42 49 84 97 144 40 76 23 89 113 160 110 96 127 151 33 46 56 2 17 28 114 141 155 100 134 83 21 86 87 98 112 31 108 4 65 149 154 47 67 132 103 109 148 37 79 85 12 32 45 64 111 122 133 7 150 152 77 91 143 11 63 170 69 107 121 146 102 105 159 8 36 130 71 164 54 61 104 15 156 57 93 116 120] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[330 216 293 181 228 282 253 273 285 305 231 237 196 296 220 255 190 236 299 176 241 199 244 277 340 211 258 212 313 198 227 232 233 288 250 311 336 210 234 257 275 320 172 304 319 327 188 208 307 259 295 300 178 202 262 183 197 203 302 286 338 182 184 316 215 252 260 263 280 284 289 328 238 248 309 334 201 243 287 217 240 271 326 333 339 249 269 272 308 223 264 291 206 298 225 195 292 189 265 329 171 230 239 246 270 301 279 323 254 261 185 209 331 204 221 224 256 274 322 324 337 315 303 245 294 179 193 226 310 321 191 312 332 267 174 177 200 218 222 278 325 186 187 235 251 173 175 180 297 192 229 266 194 207 290 314 318 205 281 213 214 268 276 335 219 242 283 306 247 317]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:07:44.995-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656...
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:45.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:07:45.000-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:07:45.001-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:07:45.001-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:07:45.001-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:07:45.002-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:07:45.006-05:00 [INFO] Checkpointing starts in 1804.708 sec
PipelineRuntimeCtx 2017-04-25T13:07:45.006-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:07:45.006-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:07:45.006-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:07:45.012-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:07:45.012-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:07:45.012-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:07:45.015-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:07:45.015-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:07:45.015-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:07:45.017-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:07:45.021-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:07:45.044-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:07:45.054-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:07:45.054-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:07:45.054-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:07:45.063-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:07:45.063-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:07:45.063-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:07:45.070-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:07:45.134-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:07:45.143-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:07:45.143-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:07:45.143-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:07:45.164-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:07:45.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:07:45.165-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:07:45.190-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:07:45.190-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:07:45.190-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:07:45.211-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:07:45.211-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
AsyncListener 2017-04-25T13:07:45.297-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:07:45.297-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:07:45.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CheckpointManager 2017-04-25T13:07:45.376-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:07:45.380-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:07:45.380-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:07:45.381-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:07:45.406-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:07:45.408-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:07:45.408-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:07:45.409-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:07:45.409-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:07:45.409-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:07:45.409-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656 is started----------
PipelineManager 2017-04-25T13:07:45.410-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:07:45.410-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:07:45.412-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:07:45 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 217 successful
CapiNozzle 2017-04-25T13:07:45.741-05:00 [INFO] Send batch count=234
CapiNozzle 2017-04-25T13:07:45.782-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:45.782-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:07:45 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:07:45.811-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:07:45 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 227 successful
CapiNozzle 2017-04-25T13:07:45.874-05:00 [INFO] Send batch count=1
2017/04/25 13:07:45 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 133 successful
CapiNozzle 2017-04-25T13:07:45.878-05:00 [INFO] Send batch count=125
2017/04/25 13:07:45 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:07:45 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 41 successful
StatisticsManager 2017-04-25T13:07:46.102-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:07:46 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 254 successful
CapiNozzle 2017-04-25T13:07:46.140-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:07:46.150-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:07:46 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 184 successful
CapiNozzle 2017-04-25T13:07:46.222-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
2017/04/25 13:07:46 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 272 successful
CapiNozzle 2017-04-25T13:07:46.236-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
2017/04/25 13:07:46 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:07:46.368-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
CheckpointManager 2017-04-25T13:07:46.368-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:07:46 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:07:46 UPR_STREAMREQ for vb 198 successful
CapiNozzle 2017-04-25T13:07:46.457-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
DcpNozzle 2017-04-25T13:07:46.479-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:07:46.481-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
DcpNozzle 2017-04-25T13:07:46.527-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:07:46.537-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:07:46.550-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:07:46.806-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:07:46.806-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:07:46.806-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:07:46.808-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
ReplicationManager 2017-04-25T13:07:46.822-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
CapiNozzle 2017-04-25T13:07:46.868-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:07:46.880-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
PipelineManager 2017-04-25T13:07:46.889-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:07:46.891-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:07:46.896-05:00 [INFO] err_list=[{"time":"2017-04-25T13:07:46.89602897-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]
PipelineManager 2017-04-25T13:07:46.918-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:07:47.017-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:07:47.017-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656 {"Errors": "[{\"time\":\"2017-04-25T13:07:46.89602897-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 142, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 5184, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 142, "set_filtered": 0, "set_received_from_dcp": 5184, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:07:47.018-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:47.044-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:47.055-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:07:47.105-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:07:47.105-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:07:47.105-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:07:47.106-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:47.118-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:47.193-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:47.193-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:47.211-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:07:47.406-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:07:47.425-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:07:47.631-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:07:47.641-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:07:47.733-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:07:47.751-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:07:47.897-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:07:47.908-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:07:48.358-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:07:48.501-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:07:48.504-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:07:48.459-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:07:48.513-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:07:48.513-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:07:48.513-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:48.530-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:07:48.506-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:48.535-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:49.475-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:49.815-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:49.912-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:49.925-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:50.061-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:07:50.140-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:07:50.784-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:07:50.808-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:07:50.900-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:07:50.919-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:07:51.062-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:07:51.074-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:07:51.347-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:07:51.359-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:07:51.542-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:07:51.547-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:07:51.547-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:07:51.549-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:51.592-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:51.840-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:07:51.840-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:07:51.840-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:07:51.840-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:51.840-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:51.948-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:51.948-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:51.949-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:51.949-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:07:51.962-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:07:51.966-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
CapiNozzle 2017-04-25T13:07:52.384-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:07:52.398-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
CapiNozzle 2017-04-25T13:07:52.570-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:07:52.619-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:07:52.754-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:07:52.778-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
CapiNozzle 2017-04-25T13:07:53.186-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:07:53.184-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:07:53.365-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:07:56.420-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:07:56.405-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:07:56.426-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:07:56.433-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:07:56.611-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:07:56.649-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:07:56.756-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:07:56.792-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
PipelineManager 2017-04-25T13:07:57.052-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries 
PipelineManager 2017-04-25T13:07:57.170-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:07:57.172-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:07:57.173-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
PipelineRuntimeCtx 2017-04-25T13:07:57.177-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:07:57.265-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:07:57.270-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:07:57.277-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:57.269216733 -0500 CDT
CapiNozzle 2017-04-25T13:07:57.300-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:57.316-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericSupervisor 2017-04-25T13:07:57.346-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:07:57.346-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:07:57.358-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:07:57.360-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:07:57.371-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:07:57.371-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:07:57.375-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:07:57.375-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:07:57.376-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:07:57.376-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-938194656 has been stopped
CapiNozzle 2017-04-25T13:07:57.452-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
GenericPipeline 2017-04-25T13:07:57.455-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-938194656 is stopped
PipelineManager 2017-04-25T13:07:57.455-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:07:57.455-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:07:46.89602897-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:07:57.463-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:07:57.470-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
GenericPipeline 2017-04-25T13:07:57.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
GenericPipeline 2017-04-25T13:07:57.484-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
GenericPipeline 2017-04-25T13:07:57.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-938194656
AsyncListener 2017-04-25T13:07:57.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:57.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:07:57.488-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:07:57.488-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:07:57.488-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:07:57.489-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
GenericPipeline 2017-04-25T13:07:57.499-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:07:57.499-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:07:57.505-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:07:57.508-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:07:57.508-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:07:57.508-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:07:57.508-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:07:57.510-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:07:57.510-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:07:57.510-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:07:57.511-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:07:57.513-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:07:57.513-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:57.513800218 -0500 CDT
DcpNozzle 2017-04-25T13:07:57.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:07:57.513-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:57.514-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:57.50810847 -0500 CDT
DcpNozzle 2017-04-25T13:07:57.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:57.510876911 -0500 CDT
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:07:57.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:07:57.515-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:07:57.515-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:57.516-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.516-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:57.516-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.516-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:07:57.516-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:57.516-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:07:57.516-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:57.516-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:07:57.519-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:07:57.519-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:07:57.519153628 -0500 CDT
DcpNozzle 2017-04-25T13:07:57.519-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:07:57.519-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:07:57.519-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
PipelineManager 2017-04-25T13:07:57.522-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:07:46.89602897-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Start pipeline construction}
]
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:07:57.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
XDCRFactory 2017-04-25T13:07:57.639-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:07:57.652-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:07:57.652-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:07:57.793-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:07:57.797-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:07:57.801-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:07:57.801-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:07:57.801-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:07:57.801-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:07:57.801-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:07:57.802-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:07:57.806-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
GenericSupervisor 2017-04-25T13:07:57.810-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:07:57.810-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
GenericSupervisor 2017-04-25T13:07:57.810-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:57.810-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:57.810-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:57.810-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:07:57.810-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:07:57.815-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
CheckpointManager 2017-04-25T13:07:57.815-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
CapiUtils 2017-04-25T13:07:57.839-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:07:57.841-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:07:57.842-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:07:57.846-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
StatisticsManager 2017-04-25T13:07:57.853-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:07:57.854-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
XDCRFactory 2017-04-25T13:07:57.854-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:07:57.855-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143 is constructed, start it
GenericPipeline 2017-04-25T13:07:57.858-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[85 7 114 140 24 93 99 151 165 50 95 168 169 36 14 37 67 72 135 145 153 0 166 106 127 21 63 155 163 31 89 42 60 138 22 62 73 75 11 130 160 68 58 121 137 55 152 124 164 32 51 98 148 29 100 103 108 18 126 88 113 161 110 19 54 10 119 120 46 45 66 147 158 41 30 17 83 15 112 141 77 149 90 94 12 1 92 48 105 27 125 86 104 131 38 156 162 116 56 115 6 44 47 69 142 23 76 128 35 101 81 59 170 26 65 78 80 84 150 157 16 4 70 2 117 96 64 61 118 167 91 109 79 154 74 143 71 52 57 144 159 39 123 136 40 97 33 111 87 133 146 8 5 3 28 49 53 107 122 134 139 13 132 82 43 102 20 34 129 9 25] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[203 214 238 220 232 233 253 199 229 260 275 192 217 212 222 176 207 216 259 183 195 248 223 200 221 299 320 244 316 202 190 308 291 305 191 215 262 267 290 255 306 175 225 240 315 324 323 281 174 237 276 278 287 208 196 234 246 249 264 334 274 171 188 194 197 236 293 243 288 321 340 201 254 256 184 294 257 177 178 297 206 327 339 204 314 333 273 322 245 251 332 247 239 295 296 172 318 258 284 326 252 286 298 319 173 179 226 182 335 224 270 289 309 310 312 317 186 187 227 241 285 311 272 283 328 230 266 325 209 231 235 242 219 307 337 263 277 210 180 265 279 292 302 218 303 329 205 338 331 198 213 280 301 193 228 313 336 250 269 304 330 185 189 268 181 211 261 271 282 300]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[261 271 282 300 181 211 214 238 203 220 232 233 253 199 229 260 275 192 217 212 222 176 207 216 259 248 183 195 223 200 221 299 320 202 244 316 190 308 291 305 191 267 290 215 262 255 306 175 225 240 315 323 324 281 174 237 276 278 208 287 196 234 246 249 264 334 274 188 194 197 236 293 171 288 321 340 243 256 184 201 254 294 257 178 297 177 339 206 327 333 204 314 273 322 332 245 251 247 239 295 296 172 318 258 284 326 252 286 298 319 226 173 179 182 335 224 270 289 309 310 312 317 187 227 241 285 311 186 328 272 283 230 266 325 231 235 242 209 219 307 337 263 277 210 265 279 292 302 180 329 218 303 205 338 331 280 301 193 198 213 313 336 228 250 269 304 330 185 189 268] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[25 7 85 24 114 140 151 165 50 93 99 169 36 95 168 67 72 135 145 153 0 14 37 166 21 106 127 163 31 63 155 89 138 22 42 60 75 11 62 73 68 130 160 137 55 58 121 152 124 164 98 148 29 32 51 108 18 100 103 88 126 110 113 161 10 19 54 46 119 120 147 158 41 45 66 17 30 15 83 77 112 141 90 149 12 94 1 92 27 48 105 125 131 38 86 104 116 156 162 6 56 115 69 142 23 44 47 35 76 128 101 59 81 170 78 80 84 150 157 16 26 65 2 4 70 117 64 96 61 91 118 167 79 109 154 71 74 143 144 159 39 52 57 40 123 136 33 97 111 146 8 87 133 3 5 53 107 122 134 139 13 28 49 82 132 20 43 102 9 34 129]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 stats_interval:1000 replication_type:capi active:true checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:07:57.861-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143...
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:07:57.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:07:57.865-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:07:57.865-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:07:57.866-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:07:57.866-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:07:57.866-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:07:57.867-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:07:57.871-05:00 [INFO] Checkpointing starts in 1801.031 sec
PipelineRuntimeCtx 2017-04-25T13:07:57.872-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:07:57.872-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:07:57.872-05:00 [INFO] checkpointing rountine started
GenericSupervisor 2017-04-25T13:07:57.871-05:00 [INFO] monitorPipelineHealth started
ToplogyChangeDetector 2017-04-25T13:07:57.878-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:07:57.885-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:07:57.885-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:07:57.887-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:07:57.890-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:07:57.891-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:07:57.903-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:07:57.910-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:07:57.931-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:07:57.931-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:07:57.931-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:07:57.934-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:07:57.934-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:07:57.962-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:07:58.024-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
AsyncListener 2017-04-25T13:07:58.024-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:07:58.025-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:07:58.025-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:07:58.028-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:07:58.028-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:07:58.028-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:07:58.031-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:07:58.031-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:07:58.032-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:07:58.048-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:07:58.048-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:07:58.048-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:07:58.066-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:07:58.066-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:07:58.207-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:07:58.207-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:07:58.207-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:07:58.239-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:07:58.239-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:07:58.239-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:07:58.239-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143 is started----------
PipelineManager 2017-04-25T13:07:58.239-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:07:58.239-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:07:58.241-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:07:58.297-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:07:58.301-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:07:58.301-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:07:58.303-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:07:58 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 115 successful
CapiNozzle 2017-04-25T13:07:58.684-05:00 [INFO] Send batch count=161
CapiNozzle 2017-04-25T13:07:58.728-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:07:58.728-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:07:58 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 122 successful
CapiNozzle 2017-04-25T13:07:58.739-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
2017/04/25 13:07:58 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 220 successful
CapiNozzle 2017-04-25T13:07:58.782-05:00 [INFO] Send batch count=1
2017/04/25 13:07:58 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:07:58 UPR_STREAMREQ for vb 32 successful
StatisticsManager 2017-04-25T13:07:58.975-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:07:59 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 243 successful
CapiNozzle 2017-04-25T13:07:59.049-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:07:59.060-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
2017/04/25 13:07:59 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:07:59.436-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
CheckpointManager 2017-04-25T13:07:59.436-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:07:59 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:07:59.444-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:07:59.454-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
2017/04/25 13:07:59 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:07:59 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:07:59.545-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:07:59.639-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:07:59.932-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:07:59.932-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:07:59.932-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:07:59.932-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]
ReplicationManager 2017-04-25T13:07:59.932-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]
PipelineManager 2017-04-25T13:07:59.963-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:07:59.963-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:07:59.971-05:00 [INFO] err_list=[{"time":"2017-04-25T13:07:59.971711551-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]
PipelineManager 2017-04-25T13:07:59.972-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:07:59.973-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:08:00.075-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:08:00.076-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:08:00.087-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:08:00.827-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:08:00.845-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
StatisticsManager 2017-04-25T13:08:00.898-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:08:00.898-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143 {"Errors": "[{\"time\":\"2017-04-25T13:07:59.971711551-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 12, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 68, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 24591, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 68, "set_filtered": 0, "set_received_from_dcp": 24591, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:08:01.454-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:08:01.474-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:08:02.639-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:08:03.036-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:08:03.036-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:08:03.040-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:08:03.120-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:08:03.654-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:08:03.698-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:08:03.757-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:08:04.125-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:08:04.141-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:08:04.773-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:08:04.792-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:08:06.343-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:08:06.488-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:08:06.488-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:08:06.489-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:08:06.562-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:08:08.228-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:08:08.192-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:08:08.241-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:08:08.376-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:08:09.024-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:08:09.726-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:08:10.744-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:08:10.096-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries 
PipelineManager 2017-04-25T13:08:10.807-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:08:10.809-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:08:10.811-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
PipelineRuntimeCtx 2017-04-25T13:08:10.816-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:08:10.818-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:08:10.839-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:08:10.871-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:10.819436533 -0500 CDT
GenericSupervisor 2017-04-25T13:08:11.111-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:08:11.114-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:08:11.290-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:08:11.292-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:08:11.293-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:08:11.293-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:08:11.313-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:08:11.317-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:08:11.320-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:08:11.325-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-801352143 has been stopped
AsyncListener 2017-04-25T13:08:12.912-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:12.980-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:12.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:08:13.008-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
AsyncListener 2017-04-25T13:08:13.008-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:08:13.016-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:08:13.016-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:08:13.018-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:08:13.036-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.030-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:08:13.038-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:08:13.042-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:08:13.052-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericSupervisor 2017-04-25T13:08:12.913-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:08:13.057-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
AsyncListener 2017-04-25T13:08:12.950-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:12.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:08:12.874-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
AsyncListener 2017-04-25T13:08:13.117-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.135-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:12.879-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:08:13.151-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:08:13.154-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:08:13.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.168-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:13.168-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.168-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:08:13.169-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:08:13.170-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:08:13.172-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:13.172-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:08:13.197-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:13.170429435 -0500 CDT
CapiNozzle 2017-04-25T13:08:13.209-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:08:13.212-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:13.212-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:13.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:08:13.242-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:13.170014852 -0500 CDT
GenericPipeline 2017-04-25T13:08:13.242-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-801352143
GenericSupervisor 2017-04-25T13:08:13.241-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:13.252-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:08:13.269-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-801352143 is stopped
PipelineManager 2017-04-25T13:08:13.270-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
CapiNozzle 2017-04-25T13:08:13.282-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
GenericPipeline 2017-04-25T13:08:13.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:08:13.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:08:13.305-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:08:13.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
PipelineManager 2017-04-25T13:08:13.347-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:07:59.971711551-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:08:13.353-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:08:13.427-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:08:13.428-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:13.428056475 -0500 CDT
DcpNozzle 2017-04-25T13:08:13.438-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:08:13.438-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:08:13.497-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:08:13.497-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:08:13.497-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:13.497778743 -0500 CDT
DcpNozzle 2017-04-25T13:08:13.497-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:08:13.497-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:08:13.497-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:08:13.505-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:08:13.507-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:08:13.510-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:08:13.511-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:08:13.511-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:08:13.512-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:08:13.523-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:08:13.524-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:08:13.697-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:08:13.703-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:08:13.705-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
ReplicationManager 2017-04-25T13:08:13.727-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:08:13.838-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:07:59.971711551-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]}, progress={2 source nozzles have been constructed}
]
XDCRFactory 2017-04-25T13:08:13.851-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:08:13.855-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:08:13.860-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:08:13.860-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:08:13.860-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:08:13.860-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:08:13.860-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:08:13.862-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:08:13.868-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
GenericSupervisor 2017-04-25T13:08:13.877-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:08:13.878-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
GenericSupervisor 2017-04-25T13:08:13.879-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:13.879-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:13.879-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:13.879-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:13.879-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:08:13.884-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
CheckpointManager 2017-04-25T13:08:13.884-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
CapiUtils 2017-04-25T13:08:13.957-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:08:13.958-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:08:13.959-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:08:13.962-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
StatisticsManager 2017-04-25T13:08:13.975-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:08:13.976-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
XDCRFactory 2017-04-25T13:08:13.976-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:08:13.977-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 is constructed, start it
GenericPipeline 2017-04-25T13:08:13.981-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[261 266 303 340 172 282 336 198 232 276 315 211 223 321 326 226 268 301 319 311 193 224 236 291 280 183 206 222 234 330 180 255 218 221 188 251 339 182 213 215 265 331 338 195 229 248 328 178 219 304 310 329 200 201 214 244 305 210 335 185 173 199 262 287 307 253 267 327 202 205 256 179 257 298 318 322 309 332 203 175 333 252 279 269 284 177 212 292 312 174 285 294 230 272 302 171 247 289 235 208 263 241 245 306 231 260 254 275 204 242 258 194 288 313 314 334 189 274 237 239 337 196 220 191 270 286 176 192 233 250 293 217 181 273 295 320 197 207 243 324 186 277 297 308 184 187 225 238 290 317 216 240 271 278 299 227 209 228 264 296 259 300 316 249 281 190 246 323 325 283] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[103 98 99 60 74 135 162 38 85 87 150 33 37 7 68 9 134 57 170 165 169 10 109 92 93 16 52 3 111 106 164 5 46 61 69 96 102 0 19 142 4 124 139 67 2 35 28 167 12 160 18 55 8 159 107 27 40 155 75 146 53 80 66 108 130 15 44 137 47 118 77 138 72 84 117 129 151 157 51 58 79 34 50 126 54 121 31 39 114 26 29 131 23 88 127 144 95 120 132 59 105 125 43 49 152 14 97 100 147 149 62 133 20 81 64 6 25 119 45 101 148 71 122 13 83 123 168 113 143 158 36 90 24 156 112 41 86 116 89 94 17 78 48 56 22 141 104 63 161 91 153 30 82 73 140 166 1 70 110 136 11 163 65 128 145 76 115 154 32 42 21]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[145 32 42 76 115 154 21 103 60 74 98 99 135 162 33 37 38 85 87 150 7 68 9 134 57 170 10 109 165 169 16 52 92 93 3 111 106 164 5 46 0 19 61 69 96 102 4 124 142 139 2 35 67 28 167 12 18 55 160 8 159 27 40 107 75 146 155 53 80 15 44 66 108 130 47 118 137 77 138 151 157 51 58 72 84 117 129 34 50 79 54 121 126 31 39 26 29 114 23 88 131 95 120 127 144 59 105 132 43 49 125 14 97 152 100 62 133 147 149 20 81 6 25 64 45 101 119 71 122 148 13 83 123 168 36 90 113 143 158 24 156 41 86 112 89 94 116 17 48 56 78 22 141 104 63 161 30 82 91 153 1 70 73 140 166 110 136 11 65 128 163] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[259 249 281 300 316 246 323 190 325 283 172 261 266 303 340 282 336 198 232 276 315 211 223 226 268 321 326 301 319 311 236 291 193 224 280 222 234 183 206 330 180 218 255 188 221 251 339 182 213 215 265 331 338 178 195 229 248 328 329 219 304 310 305 200 201 214 244 185 210 335 173 199 262 287 307 253 267 327 202 205 256 322 179 257 298 318 309 332 203 175 333 252 279 269 284 312 177 212 292 174 285 294 302 230 272 171 247 235 289 208 263 241 245 306 231 260 254 275 204 242 258 334 194 288 313 314 189 274 239 337 237 196 220 191 270 286 176 192 233 217 250 293 320 181 273 295 197 207 243 186 324 277 297 308 225 238 184 187 290 317 240 271 216 278 227 299 296 209 228 264]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 replication_type:capi active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info filter_expression: stats_interval:1000]
CheckpointManager 2017-04-25T13:08:13.987-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071...
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:13.993-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:08:13.994-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:08:13.995-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:08:13.995-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:08:13.995-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:08:13.998-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:08:14.011-05:00 [INFO] Checkpointing starts in 1801.221 sec
PipelineRuntimeCtx 2017-04-25T13:08:14.011-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:08:14.011-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:08:14.013-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:08:14.018-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:08:14.018-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:08:14.018-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:08:14.019-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:08:14.024-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:08:14.069-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:08:14.069-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:08:14.070-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:08:14.071-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:08:14.091-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:08:14.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:08:14.109-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:08:14.139-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:08:14.139-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:08:14.150-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:08:14.228-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
CheckpointManager 2017-04-25T13:08:14.229-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:08:14.229-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:08:14.229-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:08:14.249-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:08:14.253-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:08:14.264-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:08:14.292-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:08:14.296-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:08:14.296-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:08:14.318-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:08:14.318-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:08:14.470-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:08:14.703-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:08:14.832-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:08:14.924-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
GenericPipeline 2017-04-25T13:08:14.930-05:00 [INFO] All parts has been started
DcpNozzle 2017-04-25T13:08:14.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:08:14.910-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:08:15.066-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 is started----------
PipelineManager 2017-04-25T13:08:15.151-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:08:15.151-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:08:15.153-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:08:15.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:08:15.385-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:08:15.411-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:08:15.362-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:08:15.431-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:08:15 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:08:15 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 103 successful
CapiNozzle 2017-04-25T13:08:16.097-05:00 [INFO] Send batch count=496
CapiNozzle 2017-04-25T13:08:16.023-05:00 [INFO] Send batch count=147
2017/04/25 13:08:16 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:08:16 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:08:22 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:08:22 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:08:22 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:08:22 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:08:22 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:08:22 UPR_STREAMREQ for vb 310 successful
StatisticsManager 2017-04-25T13:08:22.945-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:08:22.858-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:08:22 UPR_STREAMREQ for vb 11 successful
CapiNozzle 2017-04-25T13:08:22.995-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:08:23 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 12 successful
StatisticsManager 2017-04-25T13:08:23.193-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:08:23.232-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:08:23 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 211 successful
StatisticsManager 2017-04-25T13:08:23.480-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:08:23 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 120 successful
PipelineManager 2017-04-25T13:08:23.620-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:08:23.684-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:08:23 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 26 successful
CapiNozzle 2017-04-25T13:08:23.706-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:08:23.753-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:08:23.798-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:08:23 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:08:23 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 138 successful
GenericSupervisor 2017-04-25T13:08:24.030-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:08:24.045-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:07:59.971711551-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1005, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 367, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 14464, "docs_rep_queue": 9020, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 4917, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 367, "set_filtered": 0, "set_received_from_dcp": 14464, "size_rep_queue": 11932438, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 4871}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 58, "docs_opt_repd": 0, "docs_rep_queue": 6376, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 58, "size_rep_queue": 7897026, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 120, "max": 151, "mean": 150.86666666666667, "min": 150}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 309, "docs_opt_repd": 0, "docs_rep_queue": 6827, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 309, "size_rep_queue": 9169869, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 416, "max": 6235, "mean": 6232.774038461538, "min": 6231}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 678, "dcp_dispatch_time": {"count": 7536, "max": 92, "mean": 8.723, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 18087, "expiry_received_from_dcp": 0, "set_received_from_dcp": 18087}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 327, "dcp_dispatch_time": {"count": 6928, "max": 77, "mean": 9.453, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 15548, "expiry_received_from_dcp": 0, "set_received_from_dcp": 15548}}

StatisticsManager 2017-04-25T13:08:24.046-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 18087 items, sent 147 items
StatisticsManager 2017-04-25T13:08:24.046-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 15548 items, sent 496 items
StatisticsManager 2017-04-25T13:08:24.046-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:18087]
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 18330 items, sent 18330 items. streams inactive: [51 83 97 149 67 68 73 163 168 170 57 140 74 94 72 165 91 44 47 49 150 71 87 160 167 152 141 143 144 153 48 45 64 69 76 89 56 151 154 96 81 93 59 65 98 145 84 146 50 58 70 147 82 90 79 155 60 95 157 158 46 99 62 85 86 92 75 88 161 54 66 78 169 53 61 63 148 164 52 77 142 162 55 80 156 159 166]
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:15747]
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 15747 items, sent 15747 items. streams inactive: [272 184 265 257 298 188 192 262 267 282 179 261 289 295 337 171 196 247 259 271 182 242 251 266 275 284 291 246 279 292 178 193 264 183 237 268 297 174 199 283 252 243 180 249 181 281 287 288 338 255 277 299 176 241 250 256 286 185 236 187 189 177 245 248 260 278 290 294 195 240 244 198 173 258 274 186 191 280 269 276 175 293 239 254 172 194 340 238 253 270 273 296 190 263 339 197 285]
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:08:24.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
2017/04/25 13:08:24 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 337 successful
CapiNozzle 2017-04-25T13:08:24.207-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:08:24 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 339 successful
CapiNozzle 2017-04-25T13:08:24.257-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
ToplogyChangeDetector 2017-04-25T13:08:24.303-05:00 [INFO] validateSourceTopology...
2017/04/25 13:08:24 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 145 successful
CapiNozzle 2017-04-25T13:08:24.365-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:08:24 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 244 successful
StatisticsManager 2017-04-25T13:08:24.380-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:08:24.421-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:08:24 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 159 successful
CapiNozzle 2017-04-25T13:08:24.615-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:08:24.615-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:08:24.615-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:08:24.616-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
2017/04/25 13:08:24 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 71 successful
ReplicationManager 2017-04-25T13:08:24.649-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:08:24.769-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:08:24.779-05:00 [INFO] Send batch count=500
2017/04/25 13:08:24 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:08:24 UPR_STREAMREQ for vb 183 successful
PipelineManager 2017-04-25T13:08:24.791-05:00 [INFO] err_list=[{"time":"2017-04-25T13:08:24.791242589-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:08:24.949-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:08:24.983-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:08:25 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 90 successful
StatisticsManager 2017-04-25T13:08:25.028-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:08:25.029-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 {"Errors": "[{\"time\":\"2017-04-25T13:08:24.791242589-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1005, "dcp_dispatch_time": 27, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 367, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 34077, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 367, "set_filtered": 0, "set_received_from_dcp": 34077, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:08:25.044-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:08:25.044-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:08:25 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 279 successful
CapiNozzle 2017-04-25T13:08:25.069-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
GenericSupervisor 2017-04-25T13:08:25.082-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:08:25.087-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:08:25.087-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:08:25.174-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:08:25 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:08:25.495-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:08:25 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 295 successful
CheckpointManager 2017-04-25T13:08:25.725-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
CheckpointManager 2017-04-25T13:08:25.725-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:08:25 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:08:25 UPR_STREAMREQ for vb 299 successful
DcpNozzle 2017-04-25T13:08:25.882-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:08:25.903-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:08:25.903-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:08:25.903-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:08:25.903-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:08:25.903-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:08:25.986-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:08:25.987-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:08:26.000-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:08:26.075-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:08:26.100-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:08:27.228-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:08:27.237-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:08:30.061-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:08:29.729-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
GenericSupervisor 2017-04-25T13:08:33.069-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:08:33.674-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:08:33.734-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:33.794-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:34.980-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:08:36.320-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:08:36.797-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:36.850-05:00 [INFO] Executing 0x681fb0 timed out
PipelineManager 2017-04-25T13:08:36.425-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:08:37.058-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:36.937-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:08:37.026-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:08:37.152-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:08:37.155-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:08:37.157-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
PipelineRuntimeCtx 2017-04-25T13:08:37.166-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:08:37.199-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:08:37.222-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:08:37.232-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:08:37.236-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
CapiNozzle 2017-04-25T13:08:37.261-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:08:37.265-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:08:37.265-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:08:37.346-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:08:39.063-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:39.244-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:38.929-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:08:38.969-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:08:38.959-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=3
GenericSupervisor 2017-04-25T13:08:39.086-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:38.943613325 -0500 CDT
CapiNozzle 2017-04-25T13:08:40.259-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:08:40.306-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:40.537-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:08:42.581-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
GenericSupervisor 2017-04-25T13:08:42.319-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:42.685-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CheckpointManager 2017-04-25T13:08:42.728-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:08:42.729-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:08:42.732-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:08:42.732-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:08:42.750-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 has been stopped
GenericSupervisor 2017-04-25T13:08:42.797-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:43.409-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:08:43.724-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:08:43.746-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
AsyncListener 2017-04-25T13:08:43.859-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:43.879-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:43.897-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:43.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:08:43.905-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:08:43.783-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
AsyncListener 2017-04-25T13:08:44.013-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:43.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:44.034-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.042-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.042-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.042-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.042-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:08:43.785-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
AsyncListener 2017-04-25T13:08:44.276-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:43.914-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:44.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:43.843-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:08:44.047-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:44.348-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:44.163-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:44.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:08:44.163-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:08:43.786-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
AsyncListener 2017-04-25T13:08:43.999-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.177-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.197-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:44.168-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:08:43.786-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-860556071
AsyncListener 2017-04-25T13:08:46.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:08:44.529-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:44.574-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:08:44.353-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:08:44.501-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:46.168-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:50.058-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:43.984-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:08:46.866-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:08:51.522-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:08:51.522-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:08:46.705-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:08:51.552-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:08:49.645-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:08:51.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:08:51.624-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:08:51.624-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:08:46.880-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:08:51.630-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:08:51.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:08:46.873-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:08:51.651-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:08:51.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:08:51.651-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:08:51.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:08:51.661-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
GenericPipeline 2017-04-25T13:08:51.689-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:08:51.702-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:08:51.705-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:08:51.706-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
DcpNozzle 2017-04-25T13:08:51.714-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericPipeline 2017-04-25T13:08:51.706-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:08:51.714-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:08:51.716-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:08:51.716-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:08:50.270-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:08:51.718-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:08:51.558-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:08:51.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiNozzle 2017-04-25T13:08:51.377-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:08:51.774-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:08:51.811-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:08:51.812-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:08:51.812-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:08:51.812-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
DcpNozzle 2017-04-25T13:08:51.661-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:08:51.840-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:08:51.882-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:08:51.882-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:08:51.882-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:08:51.882-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
CapiNozzle 2017-04-25T13:08:51.885-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:51.840573293 -0500 CDT
GenericSupervisor 2017-04-25T13:08:51.889-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:08:51.953-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
GenericSupervisor 2017-04-25T13:08:51.968-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:08:51.862-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:51.776090712 -0500 CDT
GenericPipeline 2017-04-25T13:08:51.973-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-860556071 is stopped
PipelineManager 2017-04-25T13:08:51.974-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
CapiNozzle 2017-04-25T13:08:51.992-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:08:52.083-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
PipelineManager 2017-04-25T13:08:52.133-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:08:24.791242589-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:08:52.139-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:08:52.142-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:08:52.142-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:52.1421228 -0500 CDT
DcpNozzle 2017-04-25T13:08:52.143-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:08:52.169-05:00 [INFO] releasing capi client
DcpNozzle 2017-04-25T13:08:52.183-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:08:52.183-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:08:52.183794976 -0500 CDT
DcpNozzle 2017-04-25T13:08:52.183-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:08:52.344-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:08:52.349-05:00 [INFO] releasing capi client
XDCRFactory 2017-04-25T13:08:52.507-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
PipelineManager 2017-04-25T13:08:52.513-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:08:24.791242589-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:08:52.516-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:08:52.516-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:08:52.648-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:08:52.654-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:08:52.658-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:08:52.658-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:08:52.658-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:08:52.658-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:08:52.659-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:08:52.660-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:08:52.666-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
GenericSupervisor 2017-04-25T13:08:52.688-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:08:52.692-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
GenericSupervisor 2017-04-25T13:08:52.703-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:52.703-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:52.703-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:52.703-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:08:52.703-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:08:52.730-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
CheckpointManager 2017-04-25T13:08:52.730-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
PipelineManager 2017-04-25T13:08:52.781-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:08:24.791242589-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Source nozzles are wired to target nozzles}
]
CapiUtils 2017-04-25T13:08:52.821-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:08:52.822-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:08:52.823-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:08:52.828-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
StatisticsManager 2017-04-25T13:08:52.841-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:08:52.843-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
XDCRFactory 2017-04-25T13:08:52.845-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:08:52.846-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792 is constructed, start it
GenericPipeline 2017-04-25T13:08:52.850-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[201 263 180 177 220 225 278 262 172 211 227 338 200 230 318 287 245 249 267 302 312 241 276 311 296 183 216 323 340 226 212 171 281 285 265 320 314 319 259 272 294 334 253 242 250 190 315 182 310 277 185 206 261 270 291 324 325 178 218 254 198 195 255 274 286 193 223 322 205 213 257 275 292 305 316 194 219 179 199 268 210 222 203 271 307 236 288 174 188 237 244 328 209 221 248 279 284 298 327 331 269 202 273 283 290 317 304 208 235 264 293 297 333 252 246 258 301 308 186 204 303 184 243 251 282 175 192 197 207 238 232 256 329 339 229 217 247 299 332 335 191 266 280 326 176 189 300 330 260 309 337 214 233 240 295 306 196 239 181 215 231 224 234 173 289 313 321 228 187 336] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[69 128 71 4 117 22 54 114 145 152 46 53 87 131 76 77 27 42 122 134 139 168 15 51 161 60 108 84 98 160 153 64 132 61 129 130 18 26 165 147 13 107 111 136 169 94 119 120 14 49 93 135 97 12 78 164 63 137 29 110 41 99 88 109 127 0 17 33 57 95 20 32 25 159 103 146 8 138 72 80 118 148 167 96 112 28 48 68 74 34 40 157 2 121 24 59 16 47 39 116 156 50 149 100 133 36 102 126 166 3 30 115 45 123 11 104 81 82 90 162 73 83 155 89 125 31 65 35 101 58 75 154 1 6 163 19 144 86 92 106 79 85 66 23 44 113 124 67 151 158 7 56 91 140 170 38 43 10 52 142 62 141 143 150 21 105 5 55 70 9 37]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[154 1 6 58 75 19 144 163 106 79 85 86 92 23 44 66 113 124 158 7 56 67 151 170 38 43 91 140 10 52 62 141 142 21 105 143 150 5 9 37 55 70 69 128 71 4 117 22 145 152 46 53 54 114 87 131 76 27 42 77 139 168 15 51 122 134 161 60 108 84 98 160 64 132 153 130 18 26 61 129 165 13 107 147 111 136 169 120 14 49 94 119 93 135 12 78 97 63 137 164 29 110 41 99 127 0 17 88 109 95 20 32 33 57 25 159 103 146 8 138 72 80 167 96 112 118 148 28 74 34 40 48 68 2 121 157 24 16 47 59 39 116 50 149 156 100 133 126 166 3 30 36 102 115 45 11 104 123 81 82 73 83 90 162 155 31 65 89 125 35 101] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[191 266 280 326 176 189 300 330 309 337 260 240 295 306 214 233 196 239 181 215 231 224 234 173 289 313 321 228 187 336 201 263 177 180 220 225 262 278 172 211 227 338 318 200 230 287 249 267 302 312 245 241 276 311 296 323 183 216 226 340 212 281 285 171 320 265 314 319 294 334 253 259 272 242 250 182 190 315 277 310 206 261 270 291 185 324 325 254 178 218 198 195 255 274 286 322 193 223 257 275 292 305 205 213 316 194 219 179 199 210 222 268 203 271 307 174 236 288 188 237 244 328 221 248 279 284 209 331 269 298 327 283 290 317 202 273 304 264 208 235 293 252 297 333 301 308 186 246 258 303 204 251 282 184 243 175 192 197 207 238 329 339 229 232 256 217 247 299 332 335]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi worker_batch_size:500 doc_batch_size_kb:2048 log_level:Info stats_interval:1000 filter_expression: active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:08:52.853-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792...
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:52.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:52.884-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:52.884-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:08:52.884-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:52.884-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:08:52.884-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:08:52.887-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:08:52.888-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:08:52.889-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:08:52.889-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:08:52.890-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:08:52.907-05:00 [INFO] Checkpointing starts in 1804.494 sec
PipelineRuntimeCtx 2017-04-25T13:08:52.907-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:08:52.908-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:08:52.912-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:08:52.913-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:08:52.913-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:08:52.914-05:00 [INFO] updateStats started
CheckpointManager 2017-04-25T13:08:52.918-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:08:52.926-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:08:52.935-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:08:52.935-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:08:52.936-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:08:52.970-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:08:52.970-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:08:52.978-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:08:52.999-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:08:52.999-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:08:53.056-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:08:53.071-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:08:53.092-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:08:53.155-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:08:53.155-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:08:53.155-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
CheckpointManager 2017-04-25T13:08:53.211-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:08:53.218-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:08:53.219-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:08:53.220-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:08:53.241-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:08:53.241-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:08:53.241-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:08:53.274-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:08:53.274-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:08:53.544-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:08:53.544-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:08:53.544-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:08:53.544-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:08:53.544-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:08:53.544-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:08:53.545-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792 is started----------
PipelineManager 2017-04-25T13:08:53.545-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:08:53.545-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:08:53.555-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:08:53.854-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:08:53.865-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:08:53.887-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:08:53.903-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
StatisticsManager 2017-04-25T13:08:53.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:08:54 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:08:54.203-05:00 [INFO] Send batch count=86
2017/04/25 13:08:54 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 305 successful
CapiNozzle 2017-04-25T13:08:54.346-05:00 [INFO] Send batch count=2
2017/04/25 13:08:54 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 209 successful
CapiNozzle 2017-04-25T13:08:54.375-05:00 [ERROR] Received malformed response from tcp connection vb=301. err=malformed HTTP response "harset=ISO-8859-1\"/>", num_bytes=300, res_buf={harset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f100/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] ma}

CapiNozzle 2017-04-25T13:08:54.375-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err Received malformed response from tcp connection vb=301. err=malformed HTTP response "harset=ISO-8859-1\"/>", num_bytes=300, res_buf={harset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f100/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] ma}
.
CapiNozzle 2017-04-25T13:08:54.395-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:08:54 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 334 successful
CapiNozzle 2017-04-25T13:08:54.556-05:00 [INFO] Send batch count=500
2017/04/25 13:08:54 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:08:54 UPR_STREAMREQ for vb 270 successful
StatisticsManager 2017-04-25T13:08:54.978-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:08:55 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 190 successful
CapiNozzle 2017-04-25T13:08:55.055-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:08:55.065-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CheckpointManager 2017-04-25T13:08:55.091-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
CheckpointManager 2017-04-25T13:08:55.091-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:08:55 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:08:55 UPR_STREAMREQ for vb 196 successful
DcpNozzle 2017-04-25T13:08:55.229-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:08:55.244-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:08:55.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:08:56.091-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:08:56.102-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
StatisticsManager 2017-04-25T13:08:56.948-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:08:57.002-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:08:57.015-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:08:57.666-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:08:57.680-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:08:57.681-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:08:57.717-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
StatisticsManager 2017-04-25T13:08:58.853-05:00 [INFO] Pipeline is no longer running, exit.
ReplicationManager 2017-04-25T13:08:59.749-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
StatisticsManager 2017-04-25T13:09:00.671-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 331, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 3, "docs_received_from_dcp": 65791, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 331, "set_filtered": 0, "set_received_from_dcp": 65791, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

PipelineManager 2017-04-25T13:09:01.222-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:09:01.287-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:09:01.229-05:00 [INFO] err_list=[{"time":"2017-04-25T13:09:01.229544911-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:09:01.351-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:09:01.527-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:09:01.577-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:09:02.105-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
GenericSupervisor 2017-04-25T13:09:02.938-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:09:02.951-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:09:03.003-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:09:03.443-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:09:05.182-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:09:04.878-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:09:05.820-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:09:05.988-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:09:08.759-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:09:09.789-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:09:09.851-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
PipelineManager 2017-04-25T13:09:11.881-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
CapiNozzle 2017-04-25T13:09:12.129-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
PipelineManager 2017-04-25T13:09:12.676-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:09:12.781-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:09:12.785-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
PipelineRuntimeCtx 2017-04-25T13:09:12.789-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:09:12.792-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:09:12.810-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:09:12.795-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:09:12.7942741 -0500 CDT
GenericSupervisor 2017-04-25T13:09:13.008-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:09:13.008-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:09:13.010-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:09:13.011-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:09:13.013-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:09:13.014-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:09:13.016-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:09:13.019-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:09:13.020-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:09:13.020-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-659009792 has been stopped
GenericPipeline 2017-04-25T13:09:13.204-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-659009792 is stopped
PipelineManager 2017-04-25T13:09:13.204-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:09:13.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:09:13.242-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:09:01.229544911-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

AsyncListener 2017-04-25T13:09:13.326-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.335-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:09:13.335-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:09:13.281-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
GenericPipeline 2017-04-25T13:09:13.287-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
GenericPipeline 2017-04-25T13:09:13.347-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.292-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:09:13.362-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:09:13.363-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:09:13.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
PipelineManager 2017-04-25T13:09:13.363-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:09:13.287-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
GenericPipeline 2017-04-25T13:09:13.383-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:09:13.383-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:09:13.386-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:09:13.287-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-659009792
GenericPipeline 2017-04-25T13:09:13.386-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:09:13.386-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:09:13.386-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:09:13.399-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:09:13.386472228 -0500 CDT
GenericPipeline 2017-04-25T13:09:13.410-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:09:13.410-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:09:13.410-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:09:13.410-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:09:13.424-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:09:13.424-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:09:13.424-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:09:13.410539306 -0500 CDT
DcpNozzle 2017-04-25T13:09:13.358-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:09:13.427-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:09:13.427-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:09:13.427-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:09:13.427-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:09:13.431-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:09:13.431-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:09:13.435-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:09:13.435-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:09:13.435-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:09:13.446-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:09:13.483-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:09:13.483-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:09:13.483196646 -0500 CDT
DcpNozzle 2017-04-25T13:09:13.483-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:09:13.483-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:09:13.483-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:09:13.483473397 -0500 CDT
GenericPipeline 2017-04-25T13:09:13.483-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:09:13.483-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:09:13.483-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:09:13.483-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:09:13.483-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
PipelineManager 2017-04-25T13:09:13.609-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:09:01.229544911-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:09:13.675-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:09:13.688-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:09:13.688-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:09:13.783-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:09:13.789-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:09:13.795-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:09:13.795-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:09:13.795-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:09:13.795-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:09:13.795-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:09:13.799-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:09:13.810-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
GenericSupervisor 2017-04-25T13:09:13.826-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:09:13.827-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
GenericSupervisor 2017-04-25T13:09:13.828-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:09:13.828-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:09:13.828-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:09:13.829-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:09:13.829-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:09:13.835-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
CheckpointManager 2017-04-25T13:09:13.835-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
ReplicationManager 2017-04-25T13:09:13.849-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
CapiUtils 2017-04-25T13:09:13.875-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:09:13.875-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:09:13.876-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:09:13.885-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
StatisticsManager 2017-04-25T13:09:13.916-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:09:13.917-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
XDCRFactory 2017-04-25T13:09:13.920-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:09:13.920-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 is constructed, start it
GenericPipeline 2017-04-25T13:09:13.933-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[281 305 248 263 187 249 338 181 185 218 277 293 177 209 237 245 236 284 172 199 239 272 183 193 201 289 290 324 229 288 212 252 287 262 195 233 256 309 321 268 274 282 211 222 231 283 294 326 180 198 328 332 178 205 208 228 308 221 235 307 226 273 280 299 296 300 301 320 295 238 314 306 213 250 257 267 325 246 315 317 303 175 219 224 291 253 173 184 241 275 194 327 230 331 217 322 292 297 323 200 216 255 335 171 316 340 206 227 254 259 260 319 240 223 330 225 243 258 192 286 266 232 251 271 336 188 244 270 302 174 247 312 311 176 189 276 207 202 261 304 214 310 179 220 339 190 210 242 313 333 318 215 234 264 298 186 279 285 191 203 278 265 329 337 334 182 196 197 204 269] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[36 67 89 112 62 32 48 99 9 42 130 12 37 78 138 154 164 2 24 141 144 3 100 113 115 16 20 80 15 41 94 120 150 5 86 1 146 156 46 71 123 132 169 58 153 159 98 109 54 63 73 103 131 30 81 122 52 118 18 68 31 158 111 119 8 128 147 11 28 84 133 137 23 49 51 10 26 77 167 170 47 70 114 127 91 108 143 25 93 105 162 79 121 14 44 55 104 50 66 69 136 13 53 34 82 135 145 72 76 27 117 142 59 125 38 157 35 61 64 163 65 160 21 88 96 56 60 95 155 161 22 29 75 7 57 166 39 40 139 168 92 33 116 151 6 45 85 101 165 129 149 17 106 124 4 43 74 87 90 102 148 152 107 110 83 97 126 0 19 134 140]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[223 330 225 192 243 258 286 266 251 271 336 232 270 302 188 244 174 247 312 311 176 276 189 207 261 202 304 214 310 220 339 179 242 313 333 190 210 215 234 264 298 318 186 279 191 203 278 285 265 329 337 196 197 204 269 334 182 248 263 281 305 249 187 181 185 218 338 277 293 177 209 237 245 172 199 236 284 183 193 201 239 272 324 289 290 212 229 288 252 287 195 233 256 262 309 321 268 274 282 211 222 231 283 180 198 294 326 328 332 178 205 208 228 308 221 235 307 226 273 280 299 296 300 301 320 295 238 314 213 306 257 267 325 250 246 315 317 175 219 224 303 291 253 173 184 241 275 327 194 230 331 217 322 292 297 323 200 216 255 335 171 316 340 206 227 254 259 260 319 240] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[65 160 96 21 88 95 155 161 56 60 22 29 75 166 7 57 139 168 39 40 92 33 116 151 85 101 165 6 45 129 149 17 74 87 90 102 106 124 4 43 148 152 107 110 126 83 97 134 0 19 140 89 112 36 67 62 99 32 48 130 9 42 78 138 154 164 12 37 141 144 2 24 113 115 3 100 80 16 20 94 120 150 15 41 5 86 156 1 146 123 46 71 132 169 159 58 153 98 109 54 103 131 63 73 122 30 81 52 118 18 68 31 158 111 119 147 8 128 84 133 137 11 28 51 23 49 77 167 170 10 26 47 127 70 114 143 91 108 25 93 105 162 79 121 14 104 44 55 69 136 50 66 13 53 135 145 34 82 72 76 142 27 117 59 125 38 157 64 163 35 61]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info replication_type:capi active:true worker_batch_size:500 doc_batch_size_kb:2048 stats_interval:1000 filter_expression: checkpoint_interval:1800 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:09:13.944-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561...
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:09:13.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
CheckpointManager 2017-04-25T13:09:13.948-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:09:13.960-05:00 [INFO] Checkpointing starts in 1803.298 sec
PipelineRuntimeCtx 2017-04-25T13:09:13.962-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:09:13.965-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:09:13.968-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:09:13.985-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:09:13.985-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:09:13.986-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:09:14.193-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:09:14.209-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:09:14.288-05:00 [INFO] ToplogyChangeDetectorSvc has started
CheckpointManager 2017-04-25T13:09:14.288-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:09:14.295-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:09:14.342-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:09:14.399-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:09:14.403-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:09:14.456-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:09:14.945-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:09:15.268-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:09:15.890-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:09:15.559-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:09:16.114-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
DcpNozzle 2017-04-25T13:09:17.238-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:09:20.990-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
GenericSupervisor 2017-04-25T13:09:21.416-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:09:22.009-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:09:22.303-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
ToplogyChangeDetector 2017-04-25T13:09:22.574-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:09:23.692-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:09:23.829-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:09:24.047-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:09:24.127-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:09:24.129-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:09:24.130-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
PipelineManager 2017-04-25T13:09:24.111-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:09:01.229544911-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:09:24.180-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:09:24.181-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:09:24.207-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
StatisticsManager 2017-04-25T13:09:24.279-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:09:24.442-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:09:24.462-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CapiNozzle 2017-04-25T13:09:24.496-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
DcpNozzle 2017-04-25T13:09:24.510-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:09:24.512-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
GenericSupervisor 2017-04-25T13:09:24.628-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:09:24.643-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:09:01.229544911-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:09:24.644-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:09:24.644-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:09:24.650-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:09:24.723-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [15 19 45 118 138 160 161 57 94 164 21 26 145 148 10 37 62 83 124 132 136 142 47 86 108 123 63 68 75 80 102 165 25 31 35 84 105 120 141 167 16 33 46 58 69 78 97 159 42 98 130 153 157 158 163 113 156 0 7 13 27 65 104 3 14 52 59 70 140 66 81 151 152 22 116 127 134 169 20 92 107 112 135 166 2 9 18 89 60 96 154 24 36 74 87 115 119 1 5 53 85 128 144 23 28 100 109 131 137 149 32 76 121 126 146 4 90 64 91 99 103 117 143 41 43 44 50 51 73 77 150 170 11 34 72 88 95 106 133 17 40 48 49 79 122 162 30 39 55 67 82 12 38 56 155 61 71 101 139 6 8 93 110 114 125 147 168 29 54 111 129]
StatisticsManager 2017-04-25T13:09:24.723-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:09:24.723-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:09:24.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
DcpNozzle 2017-04-25T13:09:25.618-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:10:09.458-05:00 [INFO] Dcp nozzle is started
DcpNozzle 2017-04-25T13:10:09.499-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericSupervisor 2017-04-25T13:09:25.667-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:10:09.522-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:10:09.526-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:10:09.540-05:00 [INFO] Pipeline is no longer running, exit.
GenericPipeline 2017-04-25T13:10:09.569-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:10:09.498-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
GenericPipeline 2017-04-25T13:10:09.593-05:00 [INFO] All parts has been started
GenericSupervisor 2017-04-25T13:10:09.695-05:00 [INFO] About to call function
CheckpointManager 2017-04-25T13:10:09.884-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:10:09.899-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:10:09.920-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
GenericPipeline 2017-04-25T13:10:09.925-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 is started----------
CheckpointManager 2017-04-25T13:10:09.931-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
PipelineManager 2017-04-25T13:10:09.934-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:10:09.934-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:10:09.935-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:10:10.110-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:10:10.452-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:10:10 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 1 successful
GenericSupervisor 2017-04-25T13:10:10.677-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:10:10.677-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:10:10.677-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:10:10 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:10:10.841-05:00 [INFO] Send batch count=152
CapiNozzle 2017-04-25T13:10:10.851-05:00 [INFO] Send batch count=250
2017/04/25 13:10:10 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:10:10 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 203 successful
StatisticsManager 2017-04-25T13:10:11.032-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:10:11 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 305 successful
StatisticsManager 2017-04-25T13:10:11.196-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:09:01.229544911-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 59, "docs_opt_repd": 0, "docs_rep_queue": 691, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 59, "size_rep_queue": 961015, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 110, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 110, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 1232, "expiry_received_from_dcp": 0, "set_received_from_dcp": 1232}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 2608, "expiry_received_from_dcp": 0, "set_received_from_dcp": 2608}}

StatisticsManager 2017-04-25T13:10:11.196-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 1232 items, sent 152 items
StatisticsManager 2017-04-25T13:10:11.196-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 2608 items, sent 250 items
StatisticsManager 2017-04-25T13:10:11.199-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:1232]
StatisticsManager 2017-04-25T13:10:11.200-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 1232 items, sent 1232 items. streams inactive: [113 156 7 13 27 65 14 52 59 70 140 66 81 151 152 22 116 127 134 169 20 92 107 112 135 166 9 18 89 60 96 154 24 36 74 87 115 119 5 53 85 128 144 23 28 109 131 137 149 32 76 121 126 146 90 64 91 99 117 143 150 41 43 44 50 51 73 77 170 11 34 72 88 95 106 133 17 40 48 49 79 122 162 30 39 55 67 82 12 38 56 155 61 71 139 168 6 8 93 110 114 125 147 29 54 111 129 15 19 45 118 138 160 161 57 94 164 21 26 145 148 142 10 37 62 83 124 132 136 47 86 108 123 63 68 75 80 165 167 25 31 35 84 105 120 141 159 16 33 46 58 69 78 97 42 98 130 153 157 158 163]
StatisticsManager 2017-04-25T13:10:11.200-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:2608]
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 2608 items, sent 2608 items. streams inactive: [184 192 219 243 248 267 281 183 338 220 252 263 292 309 195 244 245 262 266 272 291 324 214 227 238 295 325 224 251 257 307 315 340 235 191 294 317 323 180 225 287 185 198 260 331 181 213 282 283 316 339 174 211 230 269 273 284 288 190 261 275 279 285 286 246 217 228 274 298 328 209 233 240 241 188 256 296 320 332 249 199 221 237 276 321 329 194 255 265 271 293 297 310 186 319 178 306 313 176 253 223 212 236 330 189 270 333 205 210 226 229 250 327 336 187 259 264 280 314 177 196 197 216 277 278 337 179 289 311 318 208 182 206 254 173 231 232 239 175 222 290 171 268 258 193 215 218 242 299 172 335 312 234 247 322 326 334 207 308]
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:11.201-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
2017/04/25 13:10:11 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 205 successful
CapiNozzle 2017-04-25T13:10:11.482-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:11.485-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:10:11.624-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:10:11 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:10:11 UPR_STREAMREQ for vb 306 successful
CapiNozzle 2017-04-25T13:10:12.117-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:10:12.153-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:12.153-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:10:12 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:10:12 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:10:12 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:10:12 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:10:12 UPR_STREAMREQ for vb 9 successful
CapiNozzle 2017-04-25T13:10:12.403-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:10:13 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:10:13 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:10:14 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:10:14 UPR_STREAMREQ for vb 110 successful
GenericSupervisor 2017-04-25T13:10:15.063-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:10:14.992-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:10:15 UPR_STREAMREQ for vb 307 successful
DcpNozzle 2017-04-25T13:10:15.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
DcpNozzle 2017-04-25T13:10:15.834-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
StatisticsManager 2017-04-25T13:10:15.921-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:09:01.229544911-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 20, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 169, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 4211, "docs_rep_queue": 691, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 4211, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 169, "set_filtered": 0, "set_received_from_dcp": 4211, "size_rep_queue": 961015, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 188}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 59, "docs_opt_repd": 0, "docs_rep_queue": 2623, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 59, "size_rep_queue": 3474432, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 122, "max": 180, "mean": 180, "min": 180}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 110, "docs_opt_repd": 0, "docs_rep_queue": 3674, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 110, "size_rep_queue": 4385407, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 208, "max": 195, "mean": 192.87980769230768, "min": 192}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 211, "dcp_dispatch_time": {"count": 1603, "max": 6784, "mean": 25.512, "min": 6}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 3025, "expiry_received_from_dcp": 0, "set_received_from_dcp": 3025}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 22, "dcp_dispatch_time": {"count": 2608, "max": 2562, "mean": 17.036, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 4038, "expiry_received_from_dcp": 0, "set_received_from_dcp": 4038}}

StatisticsManager 2017-04-25T13:10:15.922-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 3025 items, sent 152 items
StatisticsManager 2017-04-25T13:10:15.922-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 4038 items, sent 250 items
2017/04/25 13:10:16 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:10:16 UPR_STREAMREQ for vb 208 successful
GenericSupervisor 2017-04-25T13:10:16.141-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:10:16.143-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:10:16.143-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:10:16.311-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:3060]
StatisticsManager 2017-04-25T13:10:16.315-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 3060 items, sent 3060 items. streams inactive: [117 64 91 99 143 50 51 73 77 150 41 43 44 170 88 95 133 11 34 72 49 79 122 162 17 40 48 67 82 30 39 55 155 12 38 56 139 61 71 114 125 147 168 93 129 29 54 111 118 138 160 161 15 19 45 57 94 164 148 21 26 145 83 124 132 136 142 37 62 123 47 86 80 165 63 68 75 84 120 141 167 25 31 35 58 69 78 97 159 16 33 46 153 157 158 163 42 98 130 113 156 27 65 13 59 70 140 14 52 152 66 81 151 134 169 22 116 127 112 135 166 20 92 89 18 60 96 154 87 115 119 24 36 74 85 128 144 53 131 137 149 23 28 126 146 32 76 121 90]
StatisticsManager 2017-04-25T13:10:16.316-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:4152]
StatisticsManager 2017-04-25T13:10:16.316-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 4152 items, sent 4152 items. streams inactive: [215 218 242 299 172 193 312 335 322 326 334 234 247 219 243 248 267 281 183 184 192 338 263 292 309 195 220 252 262 266 272 291 324 214 244 245 295 325 224 227 238 315 340 235 251 257 317 323 180 191 294 287 185 225 331 181 198 260 282 283 316 339 174 213 269 273 284 288 190 211 230 279 285 286 246 261 275 274 298 328 209 217 228 241 188 233 240 320 332 249 256 296 237 276 321 329 194 199 221 271 293 297 310 186 255 265 319 313 176 178 223 253 330 189 212 236 270 333 229 250 327 336 187 210 226 280 314 177 259 264 216 277 278 337 179 196 197 311 318 289 254 173 182 239 175 231 232 290 171 222 258 268]
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:16.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
2017/04/25 13:10:17 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:10:17 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:10:19 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:10:19 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:10:19 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:10:19 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:10:19 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:10:20 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:10:20 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:10:20 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:10:20 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:10:20 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:10:20 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:10:20.597-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
StatisticsManager 2017-04-25T13:10:20.672-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:10:22 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:10:22 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:10:22 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:10:25 UPR_STREAMREQ for vb 15 successful
CapiNozzle 2017-04-25T13:10:23.246-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
GenericSupervisor 2017-04-25T13:10:25.196-05:00 [INFO] About to call function
2017/04/25 13:10:26 UPR_STREAMREQ for vb 316 successful
GenericSupervisor 2017-04-25T13:10:27.065-05:00 [INFO] Executing 0x681fb0 timed out
2017/04/25 13:10:27 UPR_STREAMREQ for vb 216 successful
GenericSupervisor 2017-04-25T13:10:27.080-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:10:27.226-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:10:27 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:10:27 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:10:27 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:10:27 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:10:33 UPR_STREAMREQ for vb 218 successful
CapiNozzle 2017-04-25T13:10:33.316-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:10:32.800-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: i/o timeout
CapiNozzle 2017-04-25T13:10:33.805-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:10:33.879-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: i/o timeout.
2017/04/25 13:10:33 UPR_STREAMREQ for vb 219 successful
CapiNozzle 2017-04-25T13:10:34.168-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
StatisticsManager 2017-04-25T13:10:34.489-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:10:34 UPR_STREAMREQ for vb 319 successful
StatisticsManager 2017-04-25T13:10:34.502-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:09:01.229544911-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 107, "dcp_dispatch_time": 1212, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 169, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 8873, "docs_rep_queue": 8080, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 1286, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 169, "set_filtered": 0, "set_received_from_dcp": 8873, "size_rep_queue": 10081656, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 188}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 59, "docs_opt_repd": 0, "docs_rep_queue": 5001, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 59, "size_rep_queue": 6424222, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 122, "max": 180, "mean": 180, "min": 180}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 110, "docs_opt_repd": 0, "docs_rep_queue": 6394, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 110, "size_rep_queue": 7654296, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 208, "max": 195, "mean": 192.87980769230768, "min": 192}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 3804, "max": 970418, "mean": 1476.377, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 5599, "expiry_received_from_dcp": 0, "set_received_from_dcp": 5599}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 177, "dcp_dispatch_time": {"count": 5069, "max": 556483, "mean": 1014.133, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 6814, "expiry_received_from_dcp": 0, "set_received_from_dcp": 6814}}

StatisticsManager 2017-04-25T13:10:34.661-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 6003 items, sent 152 items
StatisticsManager 2017-04-25T13:10:34.661-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 7183 items, sent 250 items
StatisticsManager 2017-04-25T13:10:34.664-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:6003]
StatisticsManager 2017-04-25T13:10:34.666-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 6003 items, sent 6003 items. streams inactive: [91 99 64 143 43 44 50 51 73 77 150 41 170 34 72 88 95 133 40 48 49 79 122 162 39 55 67 82 30 38 56 155 71 139 61 93 125 147 168 54 129 29 19 45 138 160 161 94 164 57 26 145 148 21 37 62 83 124 132 136 142 86 123 47 68 75 80 165 63 31 35 84 120 141 167 25 33 46 58 69 78 97 159 98 130 153 157 158 163 42 156 27 65 52 59 70 140 81 151 152 66 127 134 169 22 92 135 166 20 89 96 154 60 36 74 87 24 53 85 128 144 28 131 137 149 23 76 121 126 146 32 90]
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:7183]
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 7183 items, sent 7183 items. streams inactive: [177 259 264 280 277 278 337 179 196 197 289 173 182 254 175 231 232 239 171 222 290 258 268 242 299 172 193 335 326 334 234 247 322 243 248 267 281 183 184 192 338 292 195 220 252 263 266 272 291 324 244 245 262 325 224 227 238 295 340 235 251 257 323 180 191 294 185 225 287 181 198 260 331 283 339 174 282 273 284 288 190 230 269 285 286 246 261 275 279 298 328 228 274 188 233 240 241 332 249 256 296 320 276 321 329 194 199 221 237 293 297 186 255 265 271 176 178 223 253 189 236 330 270 333 250 327 336 187 226 229]
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:10:34.667-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
GenericSupervisor 2017-04-25T13:10:34.687-05:00 [INFO] About to call function
2017/04/25 13:10:34 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:10:34 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:10:34 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:10:34 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:10:34 UPR_STREAMREQ for vb 320 successful
DcpNozzle 2017-04-25T13:10:34.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:10:34.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
2017/04/25 13:10:34 UPR_STREAMREQ for vb 121 successful
PipelineManager 2017-04-25T13:10:35.021-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:10:35 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 223 successful
CapiNozzle 2017-04-25T13:10:35.259-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:10:35.263-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:10:35.263-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:10:35.264-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:10:35.382-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
2017/04/25 13:10:35 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 21 successful
PipelineManager 2017-04-25T13:10:35.503-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:10:35.505-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:10:35.527-05:00 [INFO] err_list=[{"time":"2017-04-25T13:10:35.527079556-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:10:35.535-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:10:35 UPR_STREAMREQ for vb 22 successful
GenericSupervisor 2017-04-25T13:10:35.815-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:10:35.817-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:10:35.817-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:10:35 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:10:35 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 25 successful
StatisticsManager 2017-04-25T13:10:36.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
StatisticsManager 2017-04-25T13:10:36.424-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:10:36.427-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 {"Errors": "[{\"time\":\"2017-04-25T13:10:35.527079556-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 177, "dcp_dispatch_time": 1079, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 169, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 13186, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 169, "set_filtered": 0, "set_received_from_dcp": 13186, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

2017/04/25 13:10:36 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:10:36 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 130 successful
CapiNozzle 2017-04-25T13:10:37.426-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:10:37 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:10:37 UPR_STREAMREQ for vb 132 successful
CapiNozzle 2017-04-25T13:10:37.955-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:10:38 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:10:38 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 40 successful
CapiNozzle 2017-04-25T13:10:39.437-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:39.437-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:10:39 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:10:39 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 43 successful
CapiNozzle 2017-04-25T13:10:40.399-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
PipelineManager 2017-04-25T13:10:40.435-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:10:35.527079556-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
2017/04/25 13:10:40 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:10:40 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:10:41 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:10:42 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:10:43 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:10:44 UPR_STREAMREQ for vb 259 successful
GenericSupervisor 2017-04-25T13:10:44.820-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:10:44.820-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:10:44.822-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:10:45 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:10:45 UPR_STREAMREQ for vb 161 successful
PipelineManager 2017-04-25T13:10:45.656-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:10:45.656-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:10:45.656-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:10:45.658-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
PipelineRuntimeCtx 2017-04-25T13:10:45.658-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:10:45.659-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:10:45.679-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:10:45.681-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:10:45.659904021 -0500 CDT
GenericSupervisor 2017-04-25T13:10:45.701-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:10:45.701-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:10:45.708-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:10:45.708-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:10:45.709-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:10:45.710-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:10:45.710-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:10:45.710-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 has been stopped
GenericPipeline 2017-04-25T13:10:45.713-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-795394561 is stopped
PipelineManager 2017-04-25T13:10:45.713-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:10:45.713-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:10:35.527079556-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:10:45.713-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:10:45.717-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
GenericPipeline 2017-04-25T13:10:45.717-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
GenericPipeline 2017-04-25T13:10:45.717-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
GenericPipeline 2017-04-25T13:10:45.717-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-795394561
AsyncListener 2017-04-25T13:10:45.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:10:45.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:10:45.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:10:45.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:10:45.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:10:45.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:10:45.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:10:45.721-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:10:45.721-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:10:45.721-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:10:45.721-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:10:45.721-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:10:45.721-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:10:45.721-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:10:45.724-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:10:45.724-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:10:45.726-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=171
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=172
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=173
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=174
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=175
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=176
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=177
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=178
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=179
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=180
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=181
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=182
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=183
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=184
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=185
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=186
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=187
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=188
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=189
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=190
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=191
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=192
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=193
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=194
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=195
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=196
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=197
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=198
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] There is no active stream for vb=199
GenericPipeline 2017-04-25T13:10:45.727-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:10:45.727-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:10:45.727-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:10:45.727-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:10:45.727-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:10:45.728-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:10:45.728-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:10:45.724500355 -0500 CDT
DcpNozzle 2017-04-25T13:10:45.728-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:10:45.728-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:10:45.730-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:10:45.72784531 -0500 CDT
DcpNozzle 2017-04-25T13:10:45.730-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:10:45.730-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:10:45.730-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:10:45.730-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:10:45.731-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:10:45.731-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:10:45.731-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:10:45.731-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:10:45.731-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:10:45.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:10:45.732-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:10:45.732-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:10:45.732-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=63
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=64
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=65
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=66
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=67
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=68
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=69
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=70
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=71
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=72
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=73
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=74
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=75
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=76
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=77
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=78
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=79
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=80
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=81
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=82
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=83
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=84
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=85
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=86
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=87
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=88
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=89
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=90
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=91
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=92
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=93
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=94
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=95
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=96
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=97
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=98
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=99
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=264
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=265
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=266
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=267
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=268
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=269
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=270
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=271
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=272
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=273
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=274
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=275
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=276
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=277
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=278
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=279
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=280
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=281
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=282
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=283
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=284
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=285
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=286
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=287
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=288
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=289
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=290
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=291
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=292
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=293
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=294
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=295
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=296
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=297
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=298
DcpNozzle 2017-04-25T13:10:45.746-05:00 [INFO] There is no active stream for vb=299
DcpNozzle 2017-04-25T13:10:45.776-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:10:45.776-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:10:45.776707044 -0500 CDT
DcpNozzle 2017-04-25T13:10:45.776-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:10:45.776-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:10:45.776-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=162
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=163
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=164
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=165
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=166
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=167
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=168
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=169
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] There is no active stream for vb=170
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:10:45.778702631 -0500 CDT
DcpNozzle 2017-04-25T13:10:45.778-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:10:45.778-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:10:45.778-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:10:46.261-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:10:46.261-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:10:46.261-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:10:47.130-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 3
DcpNozzle 2017-04-25T13:10:47.130-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 3
DcpNozzle 2017-04-25T13:10:47.376-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 4
DcpNozzle 2017-04-25T13:10:47.376-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 4
DcpNozzle 2017-04-25T13:10:47.486-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 5
DcpNozzle 2017-04-25T13:10:47.487-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 5
XDCRFactory 2017-04-25T13:10:47.587-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:10:47.601-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:10:47.601-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:10:47.758-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:10:47.766-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:10:47.768-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:10:47.768-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:10:47.768-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:10:47.768-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:10:47.769-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:10:47.794-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:10:47.809-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
GenericSupervisor 2017-04-25T13:10:47.811-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:10:47.811-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
GenericSupervisor 2017-04-25T13:10:47.811-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:10:47.811-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:10:47.811-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:10:47.811-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:10:47.811-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:10:47.818-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
CheckpointManager 2017-04-25T13:10:47.818-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
CapiUtils 2017-04-25T13:10:47.906-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:10:47.906-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:10:47.906-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:10:47.910-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
StatisticsManager 2017-04-25T13:10:47.922-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:10:47.923-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
XDCRFactory 2017-04-25T13:10:47.923-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:10:47.924-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006 is constructed, start it
GenericPipeline 2017-04-25T13:10:47.925-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[89 101 40 155 25 114 125 3 99 106 124 76 42 44 2 6 129 145 27 149 50 127 9 110 121 140 41 56 81 55 108 10 66 4 92 104 132 47 139 29 30 43 83 87 126 151 7 80 34 24 58 74 14 64 62 38 117 163 13 111 39 84 161 166 17 116 159 57 134 51 60 68 70 78 138 164 8 65 113 128 131 54 21 48 95 150 22 71 147 152 167 35 46 28 148 115 170 122 118 154 85 20 93 168 169 15 26 5 105 100 144 45 18 59 102 133 137 11 77 32 96 123 160 0 67 72 37 103 12 143 98 165 130 53 91 19 158 75 79 86 141 157 23 36 97 107 153 63 69 90 31 88 94 135 82 52 61 112 49 73 109 142 156 162 33 16 119 120 136 146 1] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[309 239 293 254 272 177 175 208 237 247 200 277 295 331 338 187 300 304 196 273 278 241 245 269 282 183 194 291 313 327 189 216 263 244 255 173 206 226 262 330 223 302 308 323 266 310 171 181 252 289 193 318 210 225 288 229 306 205 221 224 265 198 227 243 301 231 250 281 212 251 336 211 220 222 285 238 186 188 228 320 185 264 268 322 214 217 292 178 199 230 260 280 333 172 321 184 276 329 339 258 233 316 275 218 257 324 215 270 298 191 279 192 242 332 328 176 203 259 287 303 197 201 209 297 307 335 219 234 236 325 249 312 283 294 314 256 190 204 207 340 246 213 253 261 271 174 248 274 286 299 334 315 319 326 179 202 305 180 240 267 296 337 182 284 290 311 195 232 317 235]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[31 69 90 82 88 94 135 52 49 61 112 162 33 73 109 142 156 146 1 16 119 120 136 89 40 101 25 155 3 114 125 76 99 106 124 2 42 44 6 27 129 145 50 149 9 127 41 110 121 140 55 56 81 108 4 10 66 47 92 104 132 139 87 126 151 7 29 30 43 83 80 24 34 58 14 74 62 64 13 38 117 163 39 111 17 84 161 166 57 116 159 51 134 138 164 8 60 68 70 78 54 65 113 128 131 21 22 48 95 150 35 71 147 152 167 28 46 115 148 122 170 85 118 154 20 15 93 168 169 5 26 100 105 45 144 137 11 18 59 102 133 32 77 0 96 123 160 37 67 72 12 103 98 143 130 165 19 53 91 158 157 23 75 79 86 141 36 97 107 63 153] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[319 326 202 305 179 337 180 240 267 296 182 284 290 311 195 232 235 317 239 309 293 177 254 272 247 175 208 237 331 338 200 277 295 304 187 300 273 196 278 241 245 269 282 291 313 327 183 194 189 216 263 244 255 206 226 262 173 330 308 323 223 302 266 310 252 289 171 181 318 193 210 225 288 205 229 306 221 224 265 301 198 227 243 231 250 281 212 251 336 285 211 220 222 238 186 188 320 228 185 264 268 322 214 217 292 280 333 178 199 230 260 172 321 339 184 276 329 258 233 316 275 218 257 324 215 270 298 191 279 192 242 328 332 287 303 176 203 259 201 209 297 197 307 335 325 219 234 236 249 312 283 294 314 256 190 204 207 246 340 213 253 261 271 299 334 174 248 274 286 315]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 optimistic_replication_threshold:256 stats_interval:1000 filter_expression: active:true checkpoint_interval:1800 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info replication_type:capi worker_batch_size:500 failure_restart_interval:10]
CheckpointManager 2017-04-25T13:10:47.925-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006...
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:10:47.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:10:47.988-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:10:47.989-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:10:47.989-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:10:47.990-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:10:47.990-05:00 [INFO] Checkpointing starts in 1804.342 sec
PipelineRuntimeCtx 2017-04-25T13:10:47.990-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:10:47.990-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:10:47.992-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:10:47.992-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:10:47.998-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:10:47.998-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:10:47.999-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:10:48.006-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:10:48.006-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:10:48.006-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:10:48.006-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:10:48.012-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:10:48.115-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:10:48.133-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:10:48.319-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:10:50.139-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:10:50.529-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:10:50.848-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:10:50.873-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:10:50.861-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:10:50.883-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:10:50.885-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
StatisticsManager 2017-04-25T13:10:51.194-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:10:51.319-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:10:51.320-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:10:51.322-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:10:51.353-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:10:51.354-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:10:51.602-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:10:51.604-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:10:51.605-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:10:51.616-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:10:51.618-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:10:51.623-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
2017/04/25 13:10:51 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:10:51.734-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:10:51.734-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:10:51.734-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:10:51.747-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:10:51.747-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:10:51.871-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:10:51.871-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:10:51.871-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:10:51.871-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:10:51.871-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:10:51.872-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:10:51.877-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006 is started----------
PipelineManager 2017-04-25T13:10:51.881-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:10:51.881-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:10:51.889-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:10:51 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:10:51.918-05:00 [INFO] Send batch count=1
2017/04/25 13:10:51 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:10:51 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:10:51 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:10:51 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:10:51.975-05:00 [INFO] Send batch count=492
2017/04/25 13:10:51 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:10:51 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:10:51 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:10:51 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:10:51.996-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:52.007-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:52.026-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:10:52 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 3 successful
StatisticsManager 2017-04-25T13:10:52.090-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:10:52 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 5 successful
CapiNozzle 2017-04-25T13:10:52.143-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:52.143-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:52.164-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:10:52 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 18 successful
CapiNozzle 2017-04-25T13:10:52.361-05:00 [INFO] Send batch count=500
2017/04/25 13:10:52 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 54 successful
PipelineManager 2017-04-25T13:10:52.737-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:10:52 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 173 successful
CapiNozzle 2017-04-25T13:10:52.919-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:10:52 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 84 successful
CapiNozzle 2017-04-25T13:10:52.959-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:10:52 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:10:52 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 285 successful
StatisticsManager 2017-04-25T13:10:53.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:10:53 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:10:53.125-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:10:53.139-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:10:53 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 98 successful
CheckpointManager 2017-04-25T13:10:53.334-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
CheckpointManager 2017-04-25T13:10:53.334-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:10:53 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:10:53 UPR_STREAMREQ for vb 298 successful
DcpNozzle 2017-04-25T13:10:53.421-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:10:53.472-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:10:53.753-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:10:53.764-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:10:53.896-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:10:53.920-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
StatisticsManager 2017-04-25T13:10:54.078-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:10:54.418-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:10:54.418-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:10:54.418-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:10:54.418-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:10:54.434-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:10:54.455-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:10:54.455-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:10:54.461-05:00 [INFO] err_list=[{"time":"2017-04-25T13:10:54.461604875-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:10:54.463-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:10:54.565-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:54.565-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:54.576-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:10:54.721-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:10:54.732-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
StatisticsManager 2017-04-25T13:10:55.008-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:10:55.009-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006 {"Errors": "[{\"time\":\"2017-04-25T13:10:54.461604875-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 597, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 4, "docs_received_from_dcp": 53433, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 597, "set_filtered": 0, "set_received_from_dcp": 53440, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:10:55.283-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:10:55.301-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:10:55.502-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:10:55.502-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:10:55.502-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:10:55.502-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:10:55.503-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:10:55.571-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:55.571-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:55.582-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:10:56.314-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:10:56.369-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:10:56.552-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:10:56.613-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:10:57.307-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:10:57.307-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:10:57.307-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:10:57.307-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:10:57.307-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:10:57.463-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:57.463-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:57.476-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:10:57.607-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:10:57.623-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
GenericSupervisor 2017-04-25T13:10:58.044-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:10:58.047-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:10:58.047-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:10:58.181-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:10:58.196-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:10:58.406-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:10:58.406-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:10:58.406-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:10:58.406-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:10:58.406-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:10:58.528-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:58.528-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:58.547-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:10:58.978-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:10:58.999-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:10:59.151-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:10:59.163-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:10:59.655-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:10:59.655-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:10:59.655-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:10:59.655-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:10:59.655-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:10:59.738-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:10:59.738-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:10:59.759-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:10:59.786-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:10:59.811-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
ToplogyChangeDetector 2017-04-25T13:11:00.298-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:11:00.540-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:11:00.560-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:11:00.801-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:11:00.801-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:11:00.801-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:11:00.801-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:11:00.801-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:11:00.876-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:11:00.876-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:11:00.920-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:11:01.282-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:11:01.293-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:11:01.659-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:11:01.679-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:11:02.091-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:11:02.091-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:11:02.091-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:11:02.091-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:11:02.091-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:11:02.170-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:11:02.170-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:11:02.181-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:11:02.452-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:11:02.463-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:11:02.857-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:11:02.868-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:11:03.308-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:11:03.309-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:11:03.309-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:11:03.309-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:11:03.309-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:11:03.445-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:11:03.445-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:11:03.456-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:11:03.678-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:11:03.689-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:11:04.013-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:11:04.024-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:11:04.421-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:11:04.421-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:11:04.421-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:11:04.421-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:11:04.421-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:11:04.472-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:11:04.472-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:11:04.472-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:11:04.473-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
PipelineRuntimeCtx 2017-04-25T13:11:04.473-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:11:04.474-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:11:04.474-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:11:04.476-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:04.474606234 -0500 CDT
GenericSupervisor 2017-04-25T13:11:04.477-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:11:04.478-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:11:04.478-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:11:04.478-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:11:04.478-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:11:04.478-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:11:04.478-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:11:04.479-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:11:04.479-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:11:04.479-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-768997006 has been stopped
GenericPipeline 2017-04-25T13:11:04.479-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-768997006 is stopped
PipelineManager 2017-04-25T13:11:04.479-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:11:04.479-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:10:54.461604875-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:11:04.479-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:11:04.483-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
GenericPipeline 2017-04-25T13:11:04.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
GenericPipeline 2017-04-25T13:11:04.484-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
GenericPipeline 2017-04-25T13:11:04.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-768997006
AsyncListener 2017-04-25T13:11:04.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:04.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:04.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:11:04.487-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:11:04.487-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:11:04.487-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:11:04.487-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:11:04.487-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:11:04.489-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:11:04.489-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:11:04.489-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:11:04.489-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:11:04.489-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:11:04.489-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:11:04.489-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:04.490-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:11:04.490-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:11:04.490-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:04.489678023 -0500 CDT
DcpNozzle 2017-04-25T13:11:04.490-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:11:04.490-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:04.490051785 -0500 CDT
CapiNozzle 2017-04-25T13:11:04.491-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:11:04.491-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:11:04.491-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:11:04.491-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:11:04.491-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:04.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:11:04.492-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:04.492-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:11:04.498-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:11:04.498-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:11:04.498-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:04.498583761 -0500 CDT
DcpNozzle 2017-04-25T13:11:04.498-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:04.498627271 -0500 CDT
DcpNozzle 2017-04-25T13:11:04.498-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:11:04.498-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:11:04.498-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:04.498-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:11:04.498-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:04.498-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:11:04 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:11:04 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:11:04 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:11:04 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:11:04 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:11:04 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:11:04.552-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:11:04.553-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:11:04.553-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:04.553-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:11:04.648-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:11:04.650-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:11:04.650-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:11:04.738-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:11:04.740-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:11:04.742-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:11:04.742-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:11:04.742-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:11:04.742-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:11:04.742-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:11:04.743-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:11:04.745-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
GenericSupervisor 2017-04-25T13:11:04.749-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:11:04.749-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
GenericSupervisor 2017-04-25T13:11:04.749-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:04.749-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:04.749-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:04.749-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:04.749-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:11:04.753-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
CheckpointManager 2017-04-25T13:11:04.753-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
CapiUtils 2017-04-25T13:11:04.867-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:11:04.867-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:11:04.867-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:11:04.870-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
StatisticsManager 2017-04-25T13:11:04.874-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:11:04.875-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
XDCRFactory 2017-04-25T13:11:04.875-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:11:04.876-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395 is constructed, start it
GenericPipeline 2017-04-25T13:11:04.878-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[1 41 91 100 0 131 17 21 53 170 19 119 156 39 142 168 43 86 140 75 54 120 6 13 141 8 56 112 114 117 135 7 126 138 9 63 64 158 12 149 99 52 144 124 31 76 48 108 89 143 92 88 130 164 67 34 38 125 132 161 29 60 66 87 96 153 50 68 72 85 90 137 163 24 121 139 62 157 4 77 81 36 152 10 61 71 26 45 57 79 2 146 105 74 78 103 65 22 148 42 15 111 25 133 37 18 49 55 98 101 162 11 70 95 109 32 27 147 16 116 145 97 83 3 51 73 166 46 40 47 118 23 58 122 155 107 93 106 104 160 20 113 134 33 129 115 82 151 14 35 69 80 84 123 165 30 150 159 169 94 44 59 28 154 167 127 102 128 136 5 110] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[215 262 187 228 236 172 325 196 218 224 306 326 193 266 276 213 214 269 288 301 333 189 246 295 316 331 220 229 317 238 197 211 180 253 173 200 212 328 204 248 274 337 179 199 265 336 240 284 311 182 241 249 277 303 314 183 201 291 298 312 318 242 192 239 250 315 322 230 207 175 191 216 225 320 330 285 174 258 273 286 194 251 340 280 226 234 254 256 245 255 278 300 307 257 231 235 281 205 219 184 198 261 264 292 176 195 210 217 271 275 283 323 335 294 208 232 268 329 338 244 263 267 297 310 202 309 178 243 177 221 260 272 293 279 287 171 313 327 186 223 259 308 185 247 282 319 209 302 188 190 227 339 296 332 334 270 299 324 289 181 237 233 203 252 321 206 305 222 290 304]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[174 285 258 273 286 194 251 340 280 226 234 254 256 245 255 278 300 257 307 235 281 231 205 219 198 261 184 264 292 195 210 217 271 275 283 176 335 294 323 208 232 268 329 338 244 263 267 297 310 202 309 178 243 177 221 260 272 293 287 171 279 313 186 327 223 259 308 247 282 319 185 302 209 188 190 227 339 296 332 334 270 324 289 299 181 237 233 203 252 321 206 305 290 304 222 215 262 187 228 236 172 325 196 218 224 306 326 193 266 276 213 214 269 288 301 333 189 246 295 316 331 220 229 238 317 197 211 253 180 173 200 212 328 248 274 337 204 179 199 265 336 240 284 311 182 241 249 277 303 314 183 201 298 312 318 242 291 192 250 315 322 230 239 207 175 191 216 225 320 330] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[10 152 71 26 61 57 79 2 45 105 146 78 103 65 74 22 148 15 42 111 25 37 133 49 55 98 101 162 11 18 70 109 32 95 147 16 27 145 97 116 3 83 73 166 46 51 47 118 23 40 58 155 107 122 93 106 160 20 104 134 33 113 115 129 151 14 82 69 80 84 123 165 30 35 159 169 94 150 59 28 44 167 127 154 128 136 5 102 110 41 91 100 0 1 17 131 53 170 19 21 156 39 119 168 43 142 140 75 86 120 6 54 141 8 13 112 114 117 135 7 56 138 9 126 64 158 12 63 99 149 52 124 144 31 48 76 89 108 92 143 130 164 67 88 38 125 132 161 29 34 66 87 96 153 50 60 72 85 90 137 163 24 68 139 62 121 4 157 81 36 77]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info stats_interval:1000 filter_expression: checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi active:true doc_batch_size_kb:2048 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:11:04.878-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395...
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:04.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:11:04.903-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:11:04.903-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:11:04.903-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:11:04.904-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:11:04.906-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:11:04.907-05:00 [INFO] Checkpointing starts in 1802.267 sec
PipelineRuntimeCtx 2017-04-25T13:11:04.908-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:11:04.908-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:11:04.908-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:11:04.909-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:11:04.909-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:11:04.909-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:11:04.909-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:11:04.910-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:11:04.935-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:11:04.936-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:11:04.936-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:11:04.939-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:11:04.939-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CapiUtils 2017-04-25T13:11:04.965-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:11:04.965-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:11:04.965-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:11:04.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:11:05.004-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:11:05.004-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:11:05.015-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:11:05.016-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:11:05.016-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:11:05.016-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:11:05.016-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:11:05.016-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:11:05.070-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:11:05.070-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:11:05.070-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:11:05.090-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:11:05.090-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:11:05.123-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:11:05.314-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:11:05.314-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:11:05.314-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:11:05.314-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:11:05.314-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:11:05.314-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:11:05.315-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395 is started----------
PipelineManager 2017-04-25T13:11:05.315-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:11:05.315-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:11:05.315-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:11:05.343-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:11:05.343-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:11:05.343-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:11:05.343-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:11:05 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:11:05.518-05:00 [INFO] Send batch count=11
2017/04/25 13:11:05 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 4 successful
CapiNozzle 2017-04-25T13:11:05.532-05:00 [ERROR] Received malformed response from tcp connection vb=301. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f101/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:11:05.532-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err Received malformed response from tcp connection vb=301. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f101/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:11:05.538-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:11:05.546-05:00 [ERROR] Received malformed response from tcp connection vb=103. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f202/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:11:05.546-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err Received malformed response from tcp connection vb=103. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f202/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:11:05.555-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:11:05.563-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
2017/04/25 13:11:05 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 107 successful
CapiNozzle 2017-04-25T13:11:05.737-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:11:05.741-05:00 [INFO] Send batch count=500
2017/04/25 13:11:05 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 14 successful
CapiNozzle 2017-04-25T13:11:05.759-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:11:05 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:11:05 UPR_STREAMREQ for vb 318 successful
StatisticsManager 2017-04-25T13:11:05.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:11:06 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 234 successful
CapiNozzle 2017-04-25T13:11:06.049-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:11:06.064-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
2017/04/25 13:11:06 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:11:06.393-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:11:06.406-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:11:06.417-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:11:06.417-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:11:06.417-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:11:06.417-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
ReplicationManager 2017-04-25T13:11:06.417-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
2017/04/25 13:11:06 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 166 successful
PipelineManager 2017-04-25T13:11:06.446-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:11:06.446-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:11:06.450-05:00 [INFO] err_list=[{"time":"2017-04-25T13:11:06.45038744-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:11:06.450-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:11:06 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 275 successful
CapiNozzle 2017-04-25T13:11:06.585-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:11:06.586-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:11:06.608-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:11:06 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:11:06 UPR_STREAMREQ for vb 298 successful
StatisticsManager 2017-04-25T13:11:06.929-05:00 [INFO] Pipeline is no longer running, exit.
CheckpointManager 2017-04-25T13:11:07.038-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
CheckpointManager 2017-04-25T13:11:07.038-05:00 [INFO] Done with SetVBTimestamps
StatisticsManager 2017-04-25T13:11:07.073-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395 {"Errors": "[{\"time\":\"2017-04-25T13:11:06.45038744-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 251, "dcp_datach_length": 0, "dcp_dispatch_time": 15, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 326, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 3, "docs_received_from_dcp": 17098, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 326, "set_filtered": 0, "set_received_from_dcp": 17093, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]", "Status": "Pending"}

DcpNozzle 2017-04-25T13:11:08.227-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:11:08.113-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:11:08 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:11:08.260-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:11:08.174-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
2017/04/25 13:11:08 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:11:08 UPR_STREAMREQ for vb 98 successful
CapiNozzle 2017-04-25T13:11:08.480-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:11:08.662-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
GenericSupervisor 2017-04-25T13:11:17.462-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:11:17.390-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:11:19.248-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:11:19.745-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
PipelineManager 2017-04-25T13:11:18.047-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
PipelineManager 2017-04-25T13:11:19.752-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:11:19.755-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:11:19.757-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
PipelineRuntimeCtx 2017-04-25T13:11:19.762-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:11:19.843-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:11:19.851-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
GenericSupervisor 2017-04-25T13:11:19.450-05:00 [INFO] Pipeline is no longer running, exit.
CheckpointManager 2017-04-25T13:11:19.829-05:00 [INFO] Received finish signal
GenericSupervisor 2017-04-25T13:11:19.674-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:11:20.268-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:11:19.918-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CheckpointManager 2017-04-25T13:11:20.292-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:11:20.331-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:11:20.339-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:11:20.382-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:11:20.386-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:20.392-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:11:20.431-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:11:20.436-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:11:20.465-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
CapiNozzle 2017-04-25T13:11:20.599-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
GenericSupervisor 2017-04-25T13:11:20.570-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:20.396337848 -0500 CDT
GenericSupervisor 2017-04-25T13:11:21.529-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:11:21.530-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:11:21.536-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-742568395 has been stopped
GenericSupervisor 2017-04-25T13:11:21.385-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:11:21.732-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:11:21.959-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-742568395 is stopped
PipelineManager 2017-04-25T13:11:21.961-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:11:21.963-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:11:06.45038744-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:11:21.968-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:11:21.969-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
AsyncListener 2017-04-25T13:11:22.032-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:11:22.050-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:22.051-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:11:22.051-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:11:22.052-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:22.073-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.074-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:11:22.084-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:11:22.084-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:11:22.030-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
GenericPipeline 2017-04-25T13:11:22.031-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
GenericPipeline 2017-04-25T13:11:22.152-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:11:22.154-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:11:22.161-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:11:22.031-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
GenericPipeline 2017-04-25T13:11:22.169-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:11:22.169-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:11:22.173-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:11:22.173-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericPipeline 2017-04-25T13:11:22.031-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-742568395
GenericPipeline 2017-04-25T13:11:22.174-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:11:22.174-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:11:22.174-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:11:22.174-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:22.173172621 -0500 CDT
GenericPipeline 2017-04-25T13:11:22.179-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:11:22.179-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:11:22.180-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:11:22.180-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:11:22.180-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:11:22.185-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:22.180058095 -0500 CDT
CapiNozzle 2017-04-25T13:11:22.188-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:11:22.189-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:11:22.189-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:11:22.189-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:22.193-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:11:22.205-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:11:22.208-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:11:22.208-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:22.208-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:11:22.226-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:11:22.226-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:22.226448816 -0500 CDT
DcpNozzle 2017-04-25T13:11:22.226-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:11:22.226-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:22.226-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:11:22.346-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:11:22.346-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:11:22.346703768 -0500 CDT
DcpNozzle 2017-04-25T13:11:22.346-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:11:22.346-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:11:22.347-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
PipelineManager 2017-04-25T13:11:22.376-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:11:06.45038744-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:11:22.530-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:11:22.536-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:11:22.536-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
ReplicationManager 2017-04-25T13:11:22.584-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
AsyncListener 2017-04-25T13:11:22.591-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:11:22.591-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:11:22.591-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
XDCRFactory 2017-04-25T13:11:22.618-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:11:22.628-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:11:22.632-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:11:22.632-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:11:22.632-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:11:22.632-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:11:22.632-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:11:22.634-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:11:22.640-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
GenericSupervisor 2017-04-25T13:11:22.660-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:11:22.661-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
GenericSupervisor 2017-04-25T13:11:22.662-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:22.662-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:22.662-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:22.662-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:22.662-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:11:22.667-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
CheckpointManager 2017-04-25T13:11:22.667-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
CapiUtils 2017-04-25T13:11:22.719-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:11:22.724-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:11:22.725-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:11:22.731-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
StatisticsManager 2017-04-25T13:11:22.751-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:11:22.752-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
PipelineManager 2017-04-25T13:11:22.767-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:11:06.45038744-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is constructed}
]
XDCRFactory 2017-04-25T13:11:22.767-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:11:22.767-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 is constructed, start it
GenericPipeline 2017-04-25T13:11:22.773-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[124 22 42 99 13 146 160 0 67 104 150 48 87 120 94 98 6 16 66 97 132 169 163 4 44 50 58 91 116 31 85 83 125 37 55 74 11 36 19 109 112 121 28 49 73 144 105 106 128 118 75 140 166 29 117 142 82 102 123 64 71 80 159 41 113 148 114 17 20 95 157 135 8 92 158 23 88 154 155 2 38 59 60 126 165 24 47 131 153 77 52 69 151 21 40 46 115 149 156 1 9 53 45 54 84 143 90 122 167 30 7 18 129 133 65 78 111 63 76 141 170 14 96 137 138 33 34 61 139 162 27 145 43 57 89 107 39 72 130 25 56 68 164 168 26 147 152 70 86 110 32 51 93 100 108 35 103 5 136 161 12 79 101 62 119 3 10 81 127 15 134] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[208 306 262 290 202 241 225 247 176 201 328 275 312 256 243 280 325 333 186 228 260 273 317 327 334 255 315 270 318 302 322 230 259 250 199 292 324 242 181 313 195 299 310 174 194 284 226 307 235 283 329 180 217 240 246 251 337 203 184 221 224 249 257 269 210 298 330 190 288 294 331 305 340 215 277 282 211 171 295 229 253 287 285 320 175 207 212 178 189 197 222 234 278 286 323 213 252 238 172 185 179 233 266 291 309 193 236 191 209 261 301 289 316 216 231 237 244 187 223 335 319 272 279 265 338 258 276 173 177 214 245 300 303 204 274 200 239 254 297 281 296 336 182 321 188 220 304 264 308 227 232 218 263 183 267 205 206 332 219 293 314 311 326 192 196 198 248 271 268 339]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[133 65 78 111 63 76 141 170 14 96 137 138 33 34 61 139 162 27 145 43 57 89 107 39 72 130 25 56 68 164 168 26 147 152 70 86 110 32 51 93 100 108 35 103 5 136 161 12 79 101 62 119 3 10 81 127 15 134 124 22 42 99 13 146 160 0 67 104 150 48 87 120 94 98 6 16 66 97 132 169 163 4 44 50 58 91 116 31 85 83 125 37 55 74 11 36 19 109 112 121 28 49 73 144 105 106 128 118 75 140 166 29 117 142 82 102 123 64 71 80 159 41 113 148 114 17 20 95 157 135 8 92 158 23 88 154 155 2 38 59 60 126 165 24 47 131 153 77 52 69 151 21 40 46 115 149 156 1 9 53 45 54 84 143 90 122 167 30 7 18 129] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[191 209 261 301 289 316 216 231 237 244 187 223 335 319 272 279 265 338 258 276 173 177 214 245 300 303 204 274 200 239 254 297 281 296 336 182 321 188 220 304 264 308 227 232 218 263 183 267 205 206 332 219 293 314 311 326 192 196 198 248 271 268 339 208 306 262 290 202 241 225 247 176 201 328 275 312 256 243 280 325 333 186 228 260 273 317 327 334 255 315 270 318 302 322 230 259 250 199 292 324 242 181 313 195 299 310 174 194 284 226 307 235 283 329 180 217 240 246 251 337 203 184 221 224 249 257 269 210 298 330 190 288 294 331 305 340 215 277 282 211 171 295 229 253 287 285 320 175 207 212 178 189 197 222 234 278 286 323 213 252 238 172 185 179 233 266 291 309 193 236]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 source_nozzle_per_node:2 log_level:Info active:true checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression:]
CheckpointManager 2017-04-25T13:11:22.781-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546...
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:22.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:22.805-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:11:22.805-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:22.805-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:11:22.805-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
ToplogyChangeDetector 2017-04-25T13:11:22.810-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:11:22.810-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:11:22.814-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:11:22.820-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:11:22.820-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:11:22.824-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:11:22.850-05:00 [INFO] Checkpointing starts in 1801.964 sec
PipelineRuntimeCtx 2017-04-25T13:11:22.850-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:11:22.850-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:11:22.850-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:11:22.851-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:11:22.857-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:11:22.860-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:11:22.879-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:11:22.950-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:11:22.950-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:11:22.954-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:11:22.969-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:11:23.060-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:11:23.064-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:11:23.241-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:11:23.348-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:11:24.403-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:11:24.571-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:11:25.254-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:25.504-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:11:25.697-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
StatisticsManager 2017-04-25T13:11:26.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:11:26.889-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:11:26.998-05:00 [INFO] Dcp nozzle is started
DcpNozzle 2017-04-25T13:11:27.085-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
GenericPipeline 2017-04-25T13:11:27.067-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:11:27.095-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:11:27.178-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:11:27.498-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:11:27.499-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CheckpointManager 2017-04-25T13:11:27.506-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:11:27.508-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:11:27.540-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:11:27.540-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:11:27.608-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:11:28.167-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:11:31.794-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:11:32.244-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:11:33.303-05:00 [INFO] About to call function
CheckpointManager 2017-04-25T13:11:33.887-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:11:33.772-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:11:33.943-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericSupervisor 2017-04-25T13:11:34.748-05:00 [INFO] Executing 0x681fb0 timed out
CheckpointManager 2017-04-25T13:11:33.895-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
DcpNozzle 2017-04-25T13:11:35.209-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:11:35.212-05:00 [INFO] Dcp nozzle is started
GenericSupervisor 2017-04-25T13:11:35.063-05:00 [INFO] ****************************
DcpNozzle 2017-04-25T13:11:35.443-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:11:35.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericSupervisor 2017-04-25T13:11:35.731-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:11:35.529-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
GenericPipeline 2017-04-25T13:11:35.821-05:00 [INFO] All parts has been started
GenericSupervisor 2017-04-25T13:11:36.591-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
2017/04/25 13:11:36 UPR_STREAMREQ for vb 300 successful
GenericSupervisor 2017-04-25T13:11:36.996-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 2 consecutive heart beats
2017/04/25 13:11:37 UPR_STREAMREQ for vb 0 successful
GenericSupervisor 2017-04-25T13:11:37.145-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:11:37.729-05:00 [INFO] Send batch count=1
ToplogyChangeDetector 2017-04-25T13:11:38.962-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:11:40.844-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:11:39.076-05:00 [INFO] Send batch count=1
StatisticsManager 2017-04-25T13:11:41.147-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericPipeline 2017-04-25T13:11:42.089-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 is started----------
GenericSupervisor 2017-04-25T13:11:43.105-05:00 [INFO] About to call function
2017/04/25 13:11:43 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:11:43 UPR_STREAMREQ for vb 200 successful
GenericSupervisor 2017-04-25T13:11:43.002-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:11:45.541-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:11:45.460-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:11:45.501-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
GenericSupervisor 2017-04-25T13:11:45.962-05:00 [INFO] ****************************
PipelineManager 2017-04-25T13:11:45.982-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
GenericSupervisor 2017-04-25T13:11:46.077-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:11:46.572-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:11:46.210-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CapiNozzle 2017-04-25T13:11:47.517-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:11:47.224-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:11:50.929-05:00 [INFO] Send batch count=217
2017/04/25 13:11:51 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 1 successful
GenericSupervisor 2017-04-25T13:11:50.977-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:11:51.156-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:11:51 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 101 successful
StatisticsManager 2017-04-25T13:11:51.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:11:51 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 3 successful
StatisticsManager 2017-04-25T13:11:51.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:11:51 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:11:51.515-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:11:51.612-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:11:51.641-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:11:51 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 308 successful
StatisticsManager 2017-04-25T13:11:51.648-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:11:06.45038744-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1581, "dcp_dispatch_time": 1952, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 394, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 8188, "docs_rep_queue": 3635, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 7595, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 394, "set_filtered": 0, "set_received_from_dcp": 8188, "size_rep_queue": 5193268, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 328}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 89, "docs_opt_repd": 0, "docs_rep_queue": 2714, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 89, "size_rep_queue": 3914996, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 176, "max": 9779, "mean": 369.21590909090907, "min": 315}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 305, "docs_opt_repd": 0, "docs_rep_queue": 3612, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 305, "size_rep_queue": 5054040, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 417, "max": 5861, "mean": 311.76258992805754, "min": 292}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 4626, "max": 369, "mean": 11.236, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 5583, "expiry_received_from_dcp": 0, "set_received_from_dcp": 5583}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 581, "dcp_dispatch_time": {"count": 3562, "max": 3478585, "mean": 4473.051, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 5211, "expiry_received_from_dcp": 0, "set_received_from_dcp": 5211}}

StatisticsManager 2017-04-25T13:11:51.699-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 5583 items, sent 218 items
StatisticsManager 2017-04-25T13:11:51.699-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 6349 items, sent 501 items
2017/04/25 13:11:51 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 10 successful
StatisticsManager 2017-04-25T13:11:51.772-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:5583]
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 5583 items, sent 5583 items. streams inactive: [63 80 169 165 13 30 37 58 95 116 132 150 51 75 99 130 139 16 85 88 112 119 147 41 53 113 157 158 164 27 52 56 71 96 114 168 36 76 87 149 151 82 84 110 142 167 39 45 90 137 160 69 141 25 29 32 42 57 111 136 138 10 59 65 77 140 60 72 73 92 124 11 50 66 67 26 28 34 62 89 94 170 33 98 120 126 127 156 74 83 93 163 91 133 145 19 23 121 144 152 159 43 128 154 54 86 123 146 155 125 15 35 68 117 61 115 148 162 14 79 97 118 153 161 17 20 22 40 49 55 135 24 38 47 78 81 122 129 143 21 166 12 31 46 64 131 18 44 48 70 134]
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:6990]
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 6990 items, sent 6990 items. streams inactive: [209 264 226 233 265 283 184 197 228 243 249 323 238 262 185 193 196 214 240 281 290 291 319 294 314 338 340 174 215 271 273 287 324 231 250 297 242 192 198 235 171 178 207 208 257 268 318 222 277 278 298 312 211 213 234 247 252 182 295 316 332 225 263 179 216 229 236 329 177 237 313 326 339 181 187 258 286 194 299 331 195 245 251 191 210 218 253 267 293 315 328 333 190 217 220 227 280 320 325 336 172 176 183 241 246 248 261 284 173 259 274 292 327 232 266 270 272 279 317 321 337 175 180 188 223 230 239 254 288 296 335 276 289 186 269 282 285 322 334 189 219 221 275 330 212 255 260 199 224 244 256]
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:51.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:51.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
2017/04/25 13:11:51 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:11:51 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:11:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:11:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:11:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:11:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:11:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:11:52 UPR_STREAMREQ for vb 209 successful
CapiNozzle 2017-04-25T13:11:52.870-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
GenericSupervisor 2017-04-25T13:11:52.890-05:00 [INFO] About to call function
2017/04/25 13:11:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:11:53 UPR_STREAMREQ for vb 210 successful
StatisticsManager 2017-04-25T13:11:53.258-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:11:53 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:11:53 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:11:53 UPR_STREAMREQ for vb 113 successful
StatisticsManager 2017-04-25T13:11:53.398-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:11:06.45038744-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1581, "dcp_dispatch_time": 324, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 394, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 15511, "docs_rep_queue": 12899, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 7323, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 394, "set_filtered": 0, "set_received_from_dcp": 15511, "size_rep_queue": 16744452, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 328}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 89, "docs_opt_repd": 0, "docs_rep_queue": 5619, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 89, "size_rep_queue": 7359113, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 176, "max": 9779, "mean": 369.21590909090907, "min": 315}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 305, "docs_opt_repd": 0, "docs_rep_queue": 7280, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 305, "size_rep_queue": 9385339, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 417, "max": 5861, "mean": 311.76258992805754, "min": 292}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 7730, "max": 4030, "mean": 14.277, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 9304, "expiry_received_from_dcp": 0, "set_received_from_dcp": 9304}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 7781, "max": 596639, "mean": 632.994, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 10472, "expiry_received_from_dcp": 0, "set_received_from_dcp": 10472}}

CapiNozzle 2017-04-25T13:11:53.405-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:11:53.406-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
StatisticsManager 2017-04-25T13:11:53.407-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 10425 items, sent 218 items
StatisticsManager 2017-04-25T13:11:53.407-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 10787 items, sent 501 items
2017/04/25 13:11:53 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:11:53 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:11:53 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:11:53 UPR_STREAMREQ for vb 316 successful
StatisticsManager 2017-04-25T13:11:53.463-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:11540]
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 11540 items, sent 11540 items. streams inactive: [326 339 177 237 181 187 258 286 194 299 331 195 245 251 267 293 328 191 218 253 333 227 280 320 190 217 220 325 336 241 246 248 261 172 176 183 284 327 173 259 274 292 272 279 317 321 232 266 270 337 223 230 239 254 175 180 188 288 296 335 276 289 322 334 186 269 282 285 330 189 219 221 275 255 260 199 224 244 256 264 283 226 233 265 249 323 184 197 228 243 238 262 240 281 290 291 185 193 196 214 319 294 338 340 287 324 174 215 271 273 231 250 297 242 192 198 235 257 268 171 178 318 222 277 278 298 252 213 234 247 182 295 332 225 263 329 179 216 229 236]
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:10707]
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 10707 items, sent 10707 items. streams inactive: [25 29 32 42 57 136 138 59 65 77 140 60 72 73 92 124 50 66 67 26 28 34 62 89 94 170 33 98 120 126 127 156 74 83 93 163 91 133 145 19 23 121 144 152 159 43 128 154 54 86 123 146 155 125 15 35 68 117 61 115 148 162 79 97 118 153 161 17 20 22 40 49 55 135 24 38 47 78 81 122 129 143 21 166 31 46 64 131 18 44 48 70 134 63 80 169 165 30 37 58 95 116 132 150 51 75 99 130 139 16 85 88 119 147 41 53 157 158 164 27 52 56 71 96 114 168 36 76 87 149 151 82 84 142 167 39 45 90 137 160 69 141]
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:11:53.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
CapiNozzle 2017-04-25T13:11:53.489-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:11:53 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:11:55 UPR_STREAMREQ for vb 117 successful
StatisticsManager 2017-04-25T13:11:55.596-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
PipelineManager 2017-04-25T13:11:53.528-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:11:55 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:11:55 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:11:55 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:11:55 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:11:55 UPR_STREAMREQ for vb 318 successful
GenericSupervisor 2017-04-25T13:11:55.085-05:00 [INFO] Executing 0x681fb0 timed out
2017/04/25 13:12:00 UPR_STREAMREQ for vb 213 successful
GenericSupervisor 2017-04-25T13:12:01.054-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:12:02.050-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:12:02.284-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:12:02.284-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:12:02.339-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:12:02.365-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:12:02 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:12:02 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:12:03 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:12:03 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:12:03 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:12:03 UPR_STREAMREQ for vb 17 successful
GenericSupervisor 2017-04-25T13:12:03.306-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:12:03.351-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:12:05.735-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:12:05.706-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:12:07.337-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:12:06.264-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:12:07.345-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:12:07.401-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:12:07 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:12:07 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:12:07 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:12:07 UPR_STREAMREQ for vb 216 successful
DcpNozzle 2017-04-25T13:12:07.639-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
CapiNozzle 2017-04-25T13:12:28.113-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: i/o timeout
GenericSupervisor 2017-04-25T13:12:28.403-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:12:28.810-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: i/o timeout.
StatisticsManager 2017-04-25T13:12:28.835-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
DcpNozzle 2017-04-25T13:12:29.011-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
CapiNozzle 2017-04-25T13:12:29.194-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:12:29 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 321 successful
GenericSupervisor 2017-04-25T13:12:29.487-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:12:29.494-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:12:29.494-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:12:29.498-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:12:29 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 218 successful
CapiNozzle 2017-04-25T13:12:29.613-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:12:29 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:12:29 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 223 successful
StatisticsManager 2017-04-25T13:12:30.532-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:12:30 UPR_STREAMREQ for vb 327 successful
PipelineManager 2017-04-25T13:12:30.714-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
StatisticsManager 2017-04-25T13:12:30.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:12:30 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:12:30 UPR_STREAMREQ for vb 329 successful
StatisticsManager 2017-04-25T13:12:30.951-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:11:06.45038744-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1304, "dcp_dispatch_time": 362, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 394, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 42130, "docs_rep_queue": 39525, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 4687, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 394, "set_filtered": 0, "set_received_from_dcp": 42130, "size_rep_queue": 46924366, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 328}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 89, "docs_opt_repd": 0, "docs_rep_queue": 19056, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 89, "size_rep_queue": 22726546, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 176, "max": 9779, "mean": 369.21590909090907, "min": 315}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 305, "docs_opt_repd": 0, "docs_rep_queue": 22139, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 305, "size_rep_queue": 26734063, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 417, "max": 5861, "mean": 311.76258992805754, "min": 292}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 608, "dcp_dispatch_time": {"count": 19490, "max": 669178, "mean": 683.899, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 20752, "expiry_received_from_dcp": 0, "set_received_from_dcp": 20752}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 696, "dcp_dispatch_time": {"count": 22640, "max": 64627, "mean": 86.194, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 24291, "expiry_received_from_dcp": 0, "set_received_from_dcp": 24291}}

StatisticsManager 2017-04-25T13:12:30.951-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 20752 items, sent 218 items
StatisticsManager 2017-04-25T13:12:30.951-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 24291 items, sent 501 items
StatisticsManager 2017-04-25T13:12:30.951-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:20752]
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 20752 items, sent 20752 items. streams inactive: [28 34 62 89 94 170 26 156 33 98 93 163 74 83 133 145 91 144 152 159 128 154 43 86 146 155 54 68 35 148 162 61 79 97 153 161 40 49 55 135 38 47 78 81 129 143 166 46 64 131 31 48 70 134 44 169 63 80 165 37 58 95 132 150 30 99 130 139 51 75 88 147 85 157 158 164 41 53 56 71 96 168 27 52 36 76 87 149 151 142 167 82 84 90 137 160 39 45 141 69 32 42 57 136 138 29 65 77 140 59 73 92 60 72 66 67 50]
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:24291]
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 24291 items, sent 24291 items. streams inactive: [236 179 229 339 177 237 286 181 187 258 194 299 331 195 245 251 253 267 293 191 333 227 280 190 336 241 246 248 261 172 176 183 284 292 173 259 274 270 272 279 232 266 337 230 239 254 175 180 188 288 296 335 276 289 285 334 186 269 282 275 330 189 255 260 256 199 244 264 265 283 233 243 249 184 197 228 238 262 240 281 290 291 185 193 196 340 294 338 273 287 174 271 231 250 297 242 192 198 235 257 268 171 178 298 277 278 247 252 234 332 182 295 263]
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:12:30.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
DcpNozzle 2017-04-25T13:12:31.023-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:12:31.023-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
2017/04/25 13:12:31 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 331 successful
StatisticsManager 2017-04-25T13:12:31.324-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:12:31 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 229 successful
CapiNozzle 2017-04-25T13:12:31.473-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:12:31 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 28 successful
CapiNozzle 2017-04-25T13:12:31.484-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:31.489-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:12:31.489-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:12:31.490-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:12:31.493-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:12:31 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 29 successful
CapiNozzle 2017-04-25T13:12:31.733-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
PipelineManager 2017-04-25T13:12:31.765-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:12:31.766-05:00 [INFO] Send batch count=500
2017/04/25 13:12:31 UPR_STREAMREQ for vb 30 successful
PipelineManager 2017-04-25T13:12:31.770-05:00 [INFO] err_list=[{"time":"2017-04-25T13:12:31.770953479-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:12:31.779-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:12:31.901-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:12:31.901-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 {"Errors": "[{\"time\":\"2017-04-25T13:12:31.770953479-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1304, "dcp_dispatch_time": 48, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 394, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 45043, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 394, "set_filtered": 0, "set_received_from_dcp": 45043, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

2017/04/25 13:12:31 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:12:31 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 134 successful
CapiNozzle 2017-04-25T13:12:32.726-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:32.726-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:12:32 UPR_STREAMREQ for vb 33 successful
PipelineManager 2017-04-25T13:12:32.788-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:12:31.770953479-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
2017/04/25 13:12:32 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:12:32 UPR_STREAMREQ for vb 235 successful
GenericSupervisor 2017-04-25T13:12:32.933-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:12:32.933-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:12:32.955-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:12:33.065-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:12:33 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 35 successful
CapiNozzle 2017-04-25T13:12:33.793-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:12:33.793-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:12:33.793-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:12:33.793-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:33.794-05:00 [INFO] Send batch count=500
2017/04/25 13:12:33 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:12:33 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 142 successful
CapiNozzle 2017-04-25T13:12:34.730-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:34.732-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:12:34 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:12:34 UPR_STREAMREQ for vb 244 successful
CapiNozzle 2017-04-25T13:12:35.174-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
ReplicationManager 2017-04-25T13:12:35.205-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:12:35.255-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:12:35 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 44 successful
CapiNozzle 2017-04-25T13:12:35.488-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:12:35 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 148 successful
DcpNozzle 2017-04-25T13:12:35.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 3
DcpNozzle 2017-04-25T13:12:35.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 3
2017/04/25 13:12:35 UPR_STREAMREQ for vb 249 successful
DcpNozzle 2017-04-25T13:12:35.782-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 4
DcpNozzle 2017-04-25T13:12:35.782-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 4
2017/04/25 13:12:35 UPR_STREAMREQ for vb 250 successful
DcpNozzle 2017-04-25T13:12:35.826-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 5
2017/04/25 13:12:35 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:12:35 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:12:36.291-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:12:36 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 258 successful
CapiNozzle 2017-04-25T13:12:36.346-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:12:36 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 163 successful
CapiNozzle 2017-04-25T13:12:36.482-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:36.545-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:12:36 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:12:36 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 285 successful
CapiNozzle 2017-04-25T13:12:37.141-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:37.141-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:12:37.141-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:12:37.141-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:37.142-05:00 [INFO] Send batch count=500
2017/04/25 13:12:37 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 83 successful
CapiNozzle 2017-04-25T13:12:37.369-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:12:37.448-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:12:37 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 190 successful
CapiNozzle 2017-04-25T13:12:37.967-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:37.967-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:12:37 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:12:37 UPR_STREAMREQ for vb 86 successful
CapiNozzle 2017-04-25T13:12:38.043-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:12:38 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 195 successful
CheckpointManager 2017-04-25T13:12:38.139-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
CheckpointManager 2017-04-25T13:12:38.139-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:12:38 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 94 successful
PipelineManager 2017-04-25T13:12:38.214-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:12:31.770953479-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
2017/04/25 13:12:38 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 96 successful
DcpNozzle 2017-04-25T13:12:38.249-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:12:38.249-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:12:38 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:12:38 UPR_STREAMREQ for vb 199 successful
CapiNozzle 2017-04-25T13:12:38.508-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:12:38.508-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:12:38.508-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:12:38.508-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:38.508-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:38.997-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:38.997-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:39.081-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:12:39.226-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:12:39.282-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:12:40.677-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:12:40.700-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:12:40.802-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:12:40.843-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:12:41.319-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:12:41.362-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:12:41.532-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:12:41.532-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:12:41.532-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:12:41.532-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:41.532-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:41.721-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:41.721-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:41.746-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
PipelineManager 2017-04-25T13:12:41.820-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:12:41.820-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:12:41.820-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:12:41.823-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
PipelineRuntimeCtx 2017-04-25T13:12:41.826-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:12:41.826-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:12:41.827-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:12:41.829-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:12:41.827053377 -0500 CDT
GenericSupervisor 2017-04-25T13:12:41.834-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:12:41.834-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:12:41.834-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:12:41.834-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:12:41.834-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:12:41.834-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:12:41.834-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:12:41.835-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:12:41.835-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:12:41.835-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 has been stopped
GenericPipeline 2017-04-25T13:12:41.839-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-632765546 is stopped
PipelineManager 2017-04-25T13:12:41.839-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:12:41.839-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:12:31.770953479-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:12:41.839-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:12:41.850-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
GenericPipeline 2017-04-25T13:12:41.850-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
GenericPipeline 2017-04-25T13:12:41.850-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
GenericPipeline 2017-04-25T13:12:41.850-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-632765546
AsyncListener 2017-04-25T13:12:41.850-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:12:41.853-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:12:41.853-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:12:41.853-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:12:41.853-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:12:41.853-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:12:41.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:12:41.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:12:41.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:12:41.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:12:41.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:12:41.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:12:41.854-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:12:41.855-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:12:41.856-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:12:41.861-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:12:41.861-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:12:41.863-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:12:41.863-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:12:41.863-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:12:41.863-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:12:41.863-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:12:41.863-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:12:41.863-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:12:41.863-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:12:41.863-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:12:41.863449134 -0500 CDT
CapiNozzle 2017-04-25T13:12:41.863-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:12:41.863699047 -0500 CDT
CapiNozzle 2017-04-25T13:12:41.864-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:12:41.864-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:12:41.864-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:12:41.864-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:12:41.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:12:41.865-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:12:41.865-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:12:41.874-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:12:41.874-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:12:41.874072311 -0500 CDT
DcpNozzle 2017-04-25T13:12:41.874-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:12:41.874-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:12:41.874-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:12:41.900-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:12:41.900-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:12:41.900665871 -0500 CDT
DcpNozzle 2017-04-25T13:12:41.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:12:41.900-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:12:41.900-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:12:41.925-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:12:41.930-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:12:41.930-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:12:41.930-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:12:42.022-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:12:42.024-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:12:42.024-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:12:42.098-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:12:42.100-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:12:42.102-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:12:42.103-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:12:42.103-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:12:42.103-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:12:42.103-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:12:42.116-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:12:42.144-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
GenericSupervisor 2017-04-25T13:12:42.161-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:12:42.161-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
GenericSupervisor 2017-04-25T13:12:42.161-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:12:42.161-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:12:42.161-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:12:42.161-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:12:42.161-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:12:42.168-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
CheckpointManager 2017-04-25T13:12:42.168-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
CapiUtils 2017-04-25T13:12:42.213-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:12:42.213-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:12:42.213-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:12:42.215-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
StatisticsManager 2017-04-25T13:12:42.217-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:12:42.218-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
XDCRFactory 2017-04-25T13:12:42.218-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:12:42.218-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582 is constructed, start it
GenericPipeline 2017-04-25T13:12:42.220-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[218 305 233 245 254 260 313 246 339 196 202 221 271 219 237 253 321 279 192 238 332 191 247 292 294 176 263 185 203 235 277 229 282 338 173 194 197 278 322 215 298 240 297 324 178 186 258 264 274 309 249 257 284 207 248 220 252 317 212 179 182 187 199 287 299 280 214 239 241 286 306 329 193 195 230 283 243 308 310 210 315 336 333 285 171 227 262 289 183 300 304 273 316 206 272 276 327 208 226 211 222 255 267 334 180 234 177 198 232 188 184 251 270 244 281 307 200 236 290 318 337 225 231 172 189 213 256 326 328 331 269 301 323 190 314 320 228 311 242 204 335 250 266 302 303 216 224 296 174 265 268 275 201 340 205 223 259 291 330 175 261 288 295 312 319 181 209 293 325 217] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[47 74 94 113 45 78 119 41 79 152 0 2 20 39 35 114 127 134 150 16 30 67 156 85 145 21 32 71 96 148 48 132 157 59 133 151 18 58 76 29 37 82 164 51 75 90 93 111 162 62 23 65 24 60 84 92 135 1 42 57 81 102 126 56 83 77 167 12 142 161 91 118 26 89 108 122 27 98 100 11 8 64 101 140 143 87 40 138 159 4 50 103 129 7 72 88 73 19 44 155 166 99 33 53 86 106 130 25 68 3 6 117 124 104 160 28 34 121 128 168 15 97 125 158 131 147 22 43 95 36 38 46 80 107 112 9 31 52 63 139 141 70 5 61 144 154 105 120 13 123 137 169 116 146 163 10 54 55 115 136 149 170 14 110 17 153 69 165 49 66 109]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[208 226 222 255 267 211 334 180 234 198 177 232 188 251 270 184 244 281 307 200 236 290 318 337 225 231 189 213 256 172 326 328 331 269 301 323 320 190 314 228 311 242 204 335 303 250 266 302 224 296 216 268 174 265 275 201 340 291 330 205 223 259 312 319 175 261 288 295 181 209 293 325 217 305 218 254 260 313 233 245 246 339 221 271 196 202 321 219 237 253 279 192 238 332 294 191 247 292 176 263 235 277 185 203 338 229 282 173 194 278 322 197 215 298 240 297 324 264 274 309 178 186 258 249 257 284 207 248 220 252 317 212 179 299 182 187 199 287 280 306 329 214 239 241 286 193 195 230 283 243 308 310 315 336 210 333 285 289 171 227 262 300 304 183 273 316 206 272 276 327] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[73 19 44 155 166 99 33 53 86 106 130 25 68 3 6 117 124 104 160 28 34 121 128 168 15 97 125 158 131 147 22 43 95 112 36 38 46 80 107 141 9 31 52 63 139 70 5 61 144 154 105 120 13 123 137 169 116 146 163 10 170 54 55 115 136 149 14 110 17 153 69 165 49 66 109 47 74 94 113 45 78 119 41 79 152 0 2 20 39 35 114 127 134 150 16 30 67 156 85 145 21 32 71 96 148 48 132 157 59 133 151 18 58 76 29 37 82 164 162 51 75 90 93 111 62 23 65 24 60 84 92 135 126 1 42 57 81 102 56 83 77 167 12 142 161 91 118 26 89 108 122 27 98 100 11 8 64 101 140 143 87 40 138 159 4 50 103 129 7 72 88]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 active:true doc_batch_size_kb:2048 checkpoint_interval:1800 worker_batch_size:500 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression:]
CheckpointManager 2017-04-25T13:12:42.220-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582...
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:12:42.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:12:42.242-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:12:42.243-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:12:42.243-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:12:42.244-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:12:42.248-05:00 [INFO] Checkpointing starts in 1802.227 sec
PipelineRuntimeCtx 2017-04-25T13:12:42.248-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:12:42.248-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:12:42.248-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:12:42.248-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:12:42.265-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:12:42.265-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:12:42.265-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:12:42.265-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:12:42.273-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:12:42.284-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:12:42.284-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:12:42.284-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:12:42.315-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:12:42.365-05:00 [INFO] validateSourceTopology...
AsyncListener 2017-04-25T13:12:42.366-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:12:42.366-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:12:42.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:12:42.366-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:12:42.366-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:12:42.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
StatisticsManager 2017-04-25T13:12:42.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:12:42.488-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:12:42.623-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:12:42.623-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:12:42.623-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:12:42.623-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:12:42.678-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:12:42.678-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:12:42.678-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:12:42.698-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:12:42.698-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:12:42.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:12:42.736-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:12:42.736-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:12:42.737-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:12:42.737-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:12:42.738-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:12:42.753-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:12:42.753-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:12:42.753-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
2017/04/25 13:12:42 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 100 successful
DcpNozzle 2017-04-25T13:12:42.765-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:12:42.765-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CapiNozzle 2017-04-25T13:12:42.829-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:12:42.834-05:00 [ERROR] Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f301/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:12:42.834-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f301/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
DcpNozzle 2017-04-25T13:12:42.836-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:12:42.836-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:12:42.836-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:12:42.836-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:12:42.836-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:12:42.836-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:12:42.837-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582 is started----------
PipelineManager 2017-04-25T13:12:42.837-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:12:42.837-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:12:42.837-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CapiNozzle 2017-04-25T13:12:42.846-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:12:42 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:12:42.854-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:42.872-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:12:42.884-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:42.900-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:12:42.905-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:42.905-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:12:42.905-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:12:42.905-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:12:42.906-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:12:42.912-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:12:42.912-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:12:42.915-05:00 [INFO] err_list=[{"time":"2017-04-25T13:12:42.915243675-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:12:42.915-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:12:42 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 315 successful
CapiNozzle 2017-04-25T13:12:42.973-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:42.973-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:12:42 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:12:42 UPR_STREAMREQ for vb 125 successful
CapiNozzle 2017-04-25T13:12:42.999-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:12:43 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 74 successful
CapiNozzle 2017-04-25T13:12:43.278-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:12:43 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 265 successful
StatisticsManager 2017-04-25T13:12:43.302-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:12:43.303-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
StatisticsManager 2017-04-25T13:12:43.315-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582 {"Errors": "[{\"time\":\"2017-04-25T13:12:42.915243675-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

2017/04/25 13:12:43 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:12:43.488-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
CheckpointManager 2017-04-25T13:12:43.488-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:12:43.541-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:12:43 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:12:43 UPR_STREAMREQ for vb 298 successful
CapiNozzle 2017-04-25T13:12:43.614-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:12:43.615-05:00 [INFO] Send batch count=66
DcpNozzle 2017-04-25T13:12:43.637-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:12:43.716-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:43.724-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:43.727-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:12:43.737-05:00 [INFO] Retrying update docs for vb 209 for the 1th time
CapiNozzle 2017-04-25T13:12:43.878-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:43.890-05:00 [INFO] Retrying update docs for vb 209 for the 2th time
CapiNozzle 2017-04-25T13:12:44.057-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:12:44.057-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:12:44.057-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:12:44.057-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:44.057-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:44.076-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:44.088-05:00 [INFO] Retrying update docs for vb 209 for the 3th time
CapiNozzle 2017-04-25T13:12:44.117-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:44.117-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:44.129-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:12:44.150-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:44.150-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 209 after 3 retries
CapiNozzle 2017-04-25T13:12:44.150-05:00 [ERROR] Raise error condition batch update docs failed for vb 209 after 3 retries
GenericSupervisor 2017-04-25T13:12:44.150-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:44.150-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:44.215-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:44.215-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:44.229-05:00 [INFO] Retrying update docs for vb 209 for the 1th time
CapiNozzle 2017-04-25T13:12:44.555-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:12:44.569-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:12:44.666-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:44.677-05:00 [INFO] Retrying update docs for vb 209 for the 2th time
CapiNozzle 2017-04-25T13:12:44.931-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:12:44.949-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:12:45.135-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:45.156-05:00 [INFO] Retrying update docs for vb 209 for the 3th time
CapiNozzle 2017-04-25T13:12:45.356-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:12:45.357-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:12:45.357-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:12:45.357-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:45.357-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:45.452-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:45.452-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:45.462-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:12:45.462-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 209 after 3 retries
CapiNozzle 2017-04-25T13:12:45.462-05:00 [ERROR] Raise error condition batch update docs failed for vb 209 after 3 retries
GenericSupervisor 2017-04-25T13:12:45.462-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:45.462-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:45.466-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:12:45.558-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:45.558-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:45.574-05:00 [INFO] Retrying update docs for vb 211 for the 1th time
CapiNozzle 2017-04-25T13:12:45.866-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:12:45.878-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:12:46.025-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:12:46.067-05:00 [INFO] Retrying update docs for vb 211 for the 2th time
CapiNozzle 2017-04-25T13:12:46.167-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:12:46.179-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:12:46.379-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:12:46.405-05:00 [INFO] Retrying update docs for vb 211 for the 3th time
CapiNozzle 2017-04-25T13:12:46.587-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:12:46.587-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:12:46.587-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:12:46.588-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:46.588-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:46.661-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:46.661-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:46.687-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:12:46.725-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:12:46.725-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 211 after 3 retries
CapiNozzle 2017-04-25T13:12:46.725-05:00 [ERROR] Raise error condition batch update docs failed for vb 211 after 3 retries
GenericSupervisor 2017-04-25T13:12:46.725-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:46.725-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:46.800-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:12:46.800-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:46.815-05:00 [INFO] Retrying update docs for vb 212 for the 1th time
CapiNozzle 2017-04-25T13:12:47.382-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:12:47.481-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:12:47.515-05:00 [INFO] Retrying update docs for vb 212 for the 2th time
CapiNozzle 2017-04-25T13:12:47.570-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:12:51.501-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:12:51.738-05:00 [INFO] Retrying update docs for vb 212 for the 3th time
CapiNozzle 2017-04-25T13:12:51.938-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:12:51.960-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
GenericSupervisor 2017-04-25T13:12:52.272-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:12:52.283-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:12:52.333-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:12:52.337-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 212 after 3 retries
CapiNozzle 2017-04-25T13:12:52.337-05:00 [ERROR] Raise error condition batch update docs failed for vb 212 after 3 retries
GenericSupervisor 2017-04-25T13:12:52.315-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:12:52.338-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:12:52.351-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:12:52.431-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 215
CapiNozzle 2017-04-25T13:12:52.519-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
ToplogyChangeDetector 2017-04-25T13:12:52.551-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:12:52.551-05:00 [ERROR] batchUpdateDocs for vb 215 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:12:52.584-05:00 [INFO] Retrying update docs for vb 215 for the 1th time
PipelineManager 2017-04-25T13:12:55.052-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:12:55.848-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:12:55.321-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
PipelineManager 2017-04-25T13:12:57.835-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:12:58.726-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
CapiNozzle 2017-04-25T13:12:58.656-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
PipelineRuntimeCtx 2017-04-25T13:13:00.055-05:00 [INFO] Pipeline context is stopping...
CapiNozzle 2017-04-25T13:13:02.491-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
StatisticsManager 2017-04-25T13:13:05.591-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:13:06.933-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:13:06.937-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:13:06.942-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:13:07.004-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:13:06.911-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:13:07.320-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:13:07.220745405 -0500 CDT
GenericSupervisor 2017-04-25T13:13:07.359-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=3
GenericSupervisor 2017-04-25T13:13:07.486-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:13:07.486-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:13:07.493-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:13:07.495-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:13:07.496-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:13:07.496-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:13:07.497-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-103180582 has been stopped
GenericSupervisor 2017-04-25T13:13:07.539-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:13:07.582-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:13:07.558-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:13:07.634-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-103180582 is stopped
PipelineManager 2017-04-25T13:13:07.636-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:13:07.637-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:12:42.915243675-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:13:07.642-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:07.653-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:13:07.680-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
AsyncListener 2017-04-25T13:13:07.842-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:13:07.700-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
AsyncListener 2017-04-25T13:13:07.729-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:13:08.292-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:13:07.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:13:07.818-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:13:07.715-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
AsyncListener 2017-04-25T13:13:08.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:13:07.804-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:13:09.158-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:07.886-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:13:07.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:13:09.195-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:09.682-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:13:08.734-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:09.837-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:13:07.723-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-103180582
GenericPipeline 2017-04-25T13:13:09.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:09.898-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:13:09.204-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:13:09.898-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
GenericPipeline 2017-04-25T13:13:09.917-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:08.425-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:13:08.258-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:13:09.919-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:13:09.944-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:13:09.502-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:09.672-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:09.999-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:13:10.002-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:10.002-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:10.002-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:13:09.809-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:13:10.010-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:13:10.012-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:13:10.014-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
AsyncListener 2017-04-25T13:13:09.721-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:13:10.022-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:13:10.024-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiNozzle 2017-04-25T13:13:09.706-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:13:10.027-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:13:09.732-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:13:10.027-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:13:10.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:13:10.027-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.028-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:13:09.802-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:13:10.029-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:13:10.032-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:13:10.032-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
AsyncListener 2017-04-25T13:13:10.059-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.059-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:13:10.060-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.060-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:13:10.060-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.060-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:13:10.060-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.060-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:13:10.061-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:13:10.080-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.080-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:13:10.080-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:13:10.080-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:13:07.877-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:13:09.868-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
CapiNozzle 2017-04-25T13:13:10.048-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:13:10.027790011 -0500 CDT
AsyncListener 2017-04-25T13:13:09.948-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:13:10.322-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:13:10.716-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:13:10.695181167 -0500 CDT
CapiNozzle 2017-04-25T13:13:10.729-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:13:10.607-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:13:09.916-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:13:10.647-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
DcpNozzle 2017-04-25T13:13:10.149-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:13:10.699-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:13:10.570-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:13:10.927-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:13:11.074-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:13:11.074-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:13:11.074609388 -0500 CDT
DcpNozzle 2017-04-25T13:13:11.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:13:10.870-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:13:11.116-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:13:11.121-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:13:11.125-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
DcpNozzle 2017-04-25T13:13:11.063-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:13:11.060675021 -0500 CDT
DcpNozzle 2017-04-25T13:13:11.144-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:13:11.168-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:13:11.169-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:13:11.172-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:13:11.172-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
AsyncListener 2017-04-25T13:13:11.148-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:13:11.230-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:13:11.232-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiNozzle 2017-04-25T13:13:11.236-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:13:11.249-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:13:11.871-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:13:11.903-05:00 [INFO] releasing capi client
PipelineManager 2017-04-25T13:13:11.949-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:12:42.915243675-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:13:12.003-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:13:12.019-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:13:12.022-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
PipelineManager 2017-04-25T13:13:12.075-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:12:42.915243675-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={2 source nozzles have been constructed}
]
XDCRFactory 2017-04-25T13:13:12.164-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:13:12.172-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:13:12.180-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:13:12.180-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:13:12.180-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:13:12.180-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:13:12.181-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:13:12.187-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:13:12.198-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
GenericSupervisor 2017-04-25T13:13:12.224-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:13:12.225-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
GenericSupervisor 2017-04-25T13:13:12.226-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:12.226-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:12.227-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:12.227-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:12.227-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:13:12.235-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
CheckpointManager 2017-04-25T13:13:12.235-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
ReplicationManager 2017-04-25T13:13:12.260-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:13:12.333-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:13:12.334-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:13:12.335-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:13:12.339-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
StatisticsManager 2017-04-25T13:13:12.360-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:13:12.361-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
XDCRFactory 2017-04-25T13:13:12.363-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:13:12.363-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 is constructed, start it
GenericPipeline 2017-04-25T13:13:12.374-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[134 66 12 153 158 80 101 7 58 29 41 34 106 20 79 83 87 22 61 132 152 40 50 63 18 31 47 90 142 9 109 122 143 14 52 53 60 88 97 36 55 68 116 70 89 103 124 145 123 131 133 5 11 23 44 104 115 16 74 120 15 38 62 144 42 111 128 169 92 105 157 72 94 96 26 127 156 161 33 43 0 13 56 64 99 168 160 2 118 136 57 67 91 17 112 51 135 137 114 117 155 86 147 162 85 100 71 95 69 45 48 126 139 164 39 49 150 46 146 19 25 78 159 4 10 54 108 170 77 107 163 59 82 138 3 98 1 73 84 113 125 166 154 27 75 129 21 93 121 30 65 28 140 81 102 24 32 76 149 6 35 119 167 130 110 141 165 8 37 148 151] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[225 254 173 238 294 231 251 298 336 216 263 295 308 172 265 245 253 297 320 322 337 259 273 183 201 286 329 300 217 323 188 211 214 331 257 338 316 175 181 272 277 293 333 185 229 306 269 305 279 282 190 237 335 332 187 191 192 235 328 208 326 264 266 276 289 207 250 256 261 312 215 232 248 255 258 221 321 270 281 189 198 218 223 240 242 260 330 176 213 311 252 268 307 177 206 325 202 224 267 292 318 196 334 339 195 239 310 233 244 317 340 314 193 228 319 219 246 220 315 226 304 236 274 288 296 227 291 313 290 309 327 209 210 283 284 182 174 241 299 180 278 262 171 179 280 324 212 234 178 197 205 249 275 303 204 230 271 285 194 243 186 200 247 184 287 301 199 302 203 222]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[71 95 69 45 48 126 139 164 39 49 150 46 146 19 25 78 159 4 10 54 108 170 77 107 163 59 82 138 3 98 1 73 84 113 125 166 154 27 75 129 21 93 121 30 65 28 140 81 102 24 32 76 149 6 35 119 167 130 110 141 165 8 37 148 151 134 66 12 153 158 80 101 7 58 29 41 34 106 20 79 83 87 22 61 132 152 40 50 63 18 31 47 90 142 9 109 122 143 14 52 53 60 88 97 36 55 68 116 70 89 103 124 145 123 131 133 5 11 23 44 104 115 16 74 120 15 38 62 144 42 111 128 169 92 105 157 72 94 96 26 127 156 161 33 43 0 13 56 64 99 168 160 2 118 136 57 67 91 17 112 51 135 137 114 117 155 86 147 162 85 100] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[226 304 236 274 288 296 227 291 313 290 309 327 209 210 283 284 182 174 241 299 180 278 262 171 179 280 324 212 234 178 197 205 249 275 303 204 230 271 285 194 243 186 200 247 184 287 301 199 302 203 222 225 254 173 238 294 231 251 298 336 216 263 295 308 172 265 245 253 297 320 322 337 259 273 183 201 286 329 300 217 323 188 211 214 331 257 338 316 175 181 272 277 293 333 185 229 306 269 305 279 282 190 237 335 332 187 191 192 235 328 208 326 264 266 276 289 207 250 256 261 312 215 232 248 255 258 221 321 270 281 189 198 218 223 240 242 260 330 176 213 311 252 268 307 177 206 325 202 224 267 292 318 196 334 339 195 239 310 233 244 317 340 314 193 228 319 219 246 220 315]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 replication_type:capi]
CheckpointManager 2017-04-25T13:13:12.387-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120...
AsyncListener 2017-04-25T13:13:12.414-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:13:12.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:13:12.417-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:13:12.418-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:13:12.418-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:13:12.419-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:13:12.425-05:00 [INFO] Checkpointing starts in 1800.541 sec
PipelineRuntimeCtx 2017-04-25T13:13:12.425-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:13:12.425-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:13:12.422-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:13:12.427-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:13:12.431-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:13:12.431-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:13:12.432-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:13:12.434-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:13:12.435-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:13:12.435-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:13:12.437-05:00 [INFO] updateStats started
CheckpointManager 2017-04-25T13:13:12.452-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:13:12.453-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:13:12.476-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:13:12.488-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:13:12.489-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:13:12.489-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:13:12.506-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:13:12.510-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:13:12.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:13:12.536-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:13:12.674-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:13:12.676-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:13:12.676-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:13:12.682-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:13:12.684-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:13:12.687-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:13:12.713-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:13:12.713-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:13:12.713-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:13:12.734-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:13:12.734-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:13:12.777-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:13:12.784-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:13:12.786-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:13:12.786-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:13:12.904-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:13:12.904-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:13:12.904-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:13:12.905-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:13:12.905-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:13:12.905-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:13:12.906-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 is started----------
PipelineManager 2017-04-25T13:13:12.907-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:13:12.907-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:13:12.910-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:13:12 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:13:12 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 320 successful
CapiNozzle 2017-04-25T13:13:13.454-05:00 [INFO] Send batch count=159
2017/04/25 13:13:13 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 120 successful
CapiNozzle 2017-04-25T13:13:13.505-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:13:13.524-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:13:13 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 25 successful
CapiNozzle 2017-04-25T13:13:13.566-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:13:13.574-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:13:13 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 324 successful
CapiNozzle 2017-04-25T13:13:13.637-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:13:13 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:13:13 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:13:14 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:13:14 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:13:14 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:13:14 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:13:15 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:13:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:13:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:13:15 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:13:16.019-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:13:16.005-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:13:16.956-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:13:17.197-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:13:17 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:13:19 UPR_STREAMREQ for vb 40 successful
StatisticsManager 2017-04-25T13:13:19.144-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:13:19 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:13:19 UPR_STREAMREQ for vb 334 successful
ToplogyChangeDetector 2017-04-25T13:13:23.101-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:13:23.254-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:13:23.464-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:13:23 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:13:23 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:13:23 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:13:23 UPR_STREAMREQ for vb 228 successful
StatisticsManager 2017-04-25T13:13:23.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:13:24.071-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
GenericSupervisor 2017-04-25T13:13:26.395-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:13:25.244-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:13:28.171-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:13:28.339-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:13:28.111-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:13:28.518-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:28.968-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:13:31.551-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
GenericSupervisor 2017-04-25T13:13:30.610-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:32.600-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:33.504-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:32.930-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:13:34.072-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:13:33 UPR_STREAMREQ for vb 42 successful
GenericSupervisor 2017-04-25T13:13:34.006-05:00 [INFO] Executing 0x681fb0 timed out
2017/04/25 13:13:34 UPR_STREAMREQ for vb 336 successful
GenericSupervisor 2017-04-25T13:13:35.081-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:34.289-05:00 [INFO] ****************************
2017/04/25 13:13:36 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:13:36 UPR_STREAMREQ for vb 229 successful
GenericSupervisor 2017-04-25T13:13:35.531-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:35.971-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:40.304-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:44.158-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:13:44.533-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:45.777-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:13:46.386-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:13:45.286-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:13:47.453-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:13:48.820-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:13:48.758-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:13:49 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:13:49 UPR_STREAMREQ for vb 230 successful
GenericSupervisor 2017-04-25T13:13:49.661-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:13:49 UPR_STREAMREQ for vb 43 successful
GenericSupervisor 2017-04-25T13:13:49.894-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:13:49 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:13:49 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 145 successful
StatisticsManager 2017-04-25T13:13:50.118-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:13:50 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 147 successful
StatisticsManager 2017-04-25T13:13:50.168-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:12:42.915243675-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1999, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 391, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 44410, "docs_rep_queue": 42240, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 6716, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 391, "set_filtered": 0, "set_received_from_dcp": 44410, "size_rep_queue": 51370453, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 106}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 316, "docs_opt_repd": 0, "docs_rep_queue": 19658, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 316, "size_rep_queue": 23994823, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 417, "max": 117, "mean": 115.60911270983213, "min": 114}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 75, "docs_opt_repd": 0, "docs_rep_queue": 22582, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 75, "size_rep_queue": 27375630, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 141, "max": 78, "mean": 77.91489361702128, "min": 77}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 999, "dcp_dispatch_time": {"count": 21669, "max": 384, "mean": 11.748, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 23803, "expiry_received_from_dcp": 0, "set_received_from_dcp": 23803}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 22741, "max": 316, "mean": 11.973, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 25072, "expiry_received_from_dcp": 0, "set_received_from_dcp": 25072}}

StatisticsManager 2017-04-25T13:13:50.171-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 23803 items, sent 500 items
StatisticsManager 2017-04-25T13:13:50.172-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 25072 items, sent 159 items
StatisticsManager 2017-04-25T13:13:50.175-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:24013]
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 24013 items, sent 24013 items. streams inactive: [55 58 75 88 99 159 165 65 98 85 64 66 169 83 149 54 74 78 152 60 170 51 154 46 90 164 160 96 89 155 86 161 70 94 57 73 82 168 50 77 59 49 69 79 47 56 158 84 92 167 67 97 61 62 72 48 53 76 80 71 95 156 157 166 52 63 91 150 163 68 81 87 162 93 151 153]
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:25072]
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 25072 items, sent 25072 items. streams inactive: [242 259 298 299 173 184 197 245 290 234 267 292 277 186 235 256 266 253 241 269 288 274 183 191 193 262 250 279 294 179 247 295 194 265 275 177 180 244 273 178 188 198 287 254 257 297 174 246 260 285 187 196 238 248 251 182 249 176 270 199 281 284 286 291 181 268 264 278 280 243 172 189 289 175 195 258 239 261 282 240 296 236 237 252 283 171 192 293 190 272 185 255 263 271 276]
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =210 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =210 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:13:50.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
ToplogyChangeDetector 2017-04-25T13:13:50.209-05:00 [INFO] validateSourceTopology...
2017/04/25 13:13:50 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 236 successful
PipelineManager 2017-04-25T13:13:50.259-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:13:50 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:13:50 UPR_STREAMREQ for vb 58 successful
StatisticsManager 2017-04-25T13:13:50.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:13:52 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:13:52 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:13:52 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:13:52 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:13:52 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:13:52 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:13:52 UPR_STREAMREQ for vb 61 successful
CapiNozzle 2017-04-25T13:13:52.707-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:13:52.632-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:13:54.027-05:00 [INFO] About to call function
2017/04/25 13:13:55 UPR_STREAMREQ for vb 165 successful
GenericSupervisor 2017-04-25T13:13:57.553-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:13:58.003-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:13:57.820-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:13:58.177-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
2017/04/25 13:13:58 UPR_STREAMREQ for vb 251 successful
GenericSupervisor 2017-04-25T13:13:58.375-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:13:58.179-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:14:00.858-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
2017/04/25 13:14:02 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:14:02 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:14:02 UPR_STREAMREQ for vb 63 successful
CapiNozzle 2017-04-25T13:14:02.451-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:14:02 UPR_STREAMREQ for vb 167 successful
DcpNozzle 2017-04-25T13:14:03.311-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
GenericSupervisor 2017-04-25T13:14:03.044-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:14:07.015-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
2017/04/25 13:14:07 UPR_STREAMREQ for vb 252 successful
DcpNozzle 2017-04-25T13:14:04.942-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
GenericSupervisor 2017-04-25T13:14:06.624-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:14:08.038-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:14:08.042-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:14:08.099-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:14:06.744-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:14:09.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:14:08.156-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:14:09.905-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 3
GenericSupervisor 2017-04-25T13:14:09.925-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:14:10.565-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:14:12.274-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
GenericSupervisor 2017-04-25T13:14:12.717-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:14:12.754-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1000, "dcp_dispatch_time": 2382, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 391, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 58680, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 391, "set_filtered": 0, "set_received_from_dcp": 58680, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

PipelineManager 2017-04-25T13:14:12.541-05:00 [INFO] err_list=[{"time":"2017-04-25T13:14:12.538770796-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:14:12.821-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:14:12.822-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:14:12.723-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:14:12 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:14:12 UPR_STREAMREQ for vb 168 successful
GenericSupervisor 2017-04-25T13:14:12.865-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:14:12 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:14:12 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:14:13 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:14:13 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:14:15 UPR_STREAMREQ for vb 170 successful
ToplogyChangeDetector 2017-04-25T13:14:15.402-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:14:16 UPR_STREAMREQ for vb 65 successful
GenericSupervisor 2017-04-25T13:14:17.467-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
2017/04/25 13:14:18 UPR_STREAMREQ for vb 66 successful
GenericSupervisor 2017-04-25T13:14:18.155-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:14:18.485-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:14:19.118-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:14:19 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:14:19 UPR_STREAMREQ for vb 171 successful
CapiNozzle 2017-04-25T13:14:19.746-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:14:21 UPR_STREAMREQ for vb 67 successful
CapiNozzle 2017-04-25T13:14:20.939-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:14:21.116-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:14:21.117-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:14:21.117-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:14:21.129-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:14:22.924-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
GenericSupervisor 2017-04-25T13:14:22.761-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:14:23.033-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 2 consecutive heart beats
2017/04/25 13:14:23 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:14:23 UPR_STREAMREQ for vb 172 successful
GenericSupervisor 2017-04-25T13:14:23.041-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:14:23.042-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:14:23.059-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:14:23.062-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:14:23.063-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
PipelineRuntimeCtx 2017-04-25T13:14:23.065-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:14:23.066-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:14:23.073-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:14:23.078-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:14:23.06698192 -0500 CDT
GenericSupervisor 2017-04-25T13:14:23.084-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:14:23.084-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:14:23.089-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:14:23.090-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:14:23.092-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:14:23.093-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:14:23.094-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:14:23.094-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 has been stopped
2017/04/25 13:14:23 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:14:23 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:14:23 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:14:23 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:14:23 UPR_STREAMREQ for vb 69 successful
GenericPipeline 2017-04-25T13:14:23.293-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-181396120 is stopped
PipelineManager 2017-04-25T13:14:23.293-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:14:23.296-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:14:12.538770796-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:14:23.299-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:14:23 UPR_STREAMREQ for vb 175 successful
DcpNozzle 2017-04-25T13:14:23.340-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 4
DcpNozzle 2017-04-25T13:14:23.340-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
PipelineManager 2017-04-25T13:14:23.347-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:14:12.538770796-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Start pipeline construction}
]
2017/04/25 13:14:23 UPR_STREAMREQ for vb 70 successful
GenericPipeline 2017-04-25T13:14:23.362-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
GenericPipeline 2017-04-25T13:14:23.364-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
GenericPipeline 2017-04-25T13:14:23.364-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
GenericPipeline 2017-04-25T13:14:23.364-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-181396120
AsyncListener 2017-04-25T13:14:23.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:14:23.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:14:23.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:14:23.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:14:23.380-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:14:23.381-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:14:23.382-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:14:23.386-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:14:23.386-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:14:23.388-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:14:23.388-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:14:23.388-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:14:23.388-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:14:23.389-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:14:23.389-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:14:23.389-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:14:23.390-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:14:23.390-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:14:23.390-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:14:23.38860996 -0500 CDT
DcpNozzle 2017-04-25T13:14:23.390-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:14:23.390-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:14:23.389393019 -0500 CDT
CapiNozzle 2017-04-25T13:14:23.390-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:14:23.392-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:14:23.392-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:14:23.396-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:14:23.396-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:14:23.398-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:14:23.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:14:23.401-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:14:23.401-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=71
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=72
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=73
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=74
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=75
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=76
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=77
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=78
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=79
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=80
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=81
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=82
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=83
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=84
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=85
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=86
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=87
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=88
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=89
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=90
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=91
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=92
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=93
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=94
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=95
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=96
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=97
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=98
DcpNozzle 2017-04-25T13:14:23.402-05:00 [INFO] There is no active stream for vb=99
DcpNozzle 2017-04-25T13:14:23.414-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=176
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=177
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=178
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=179
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=180
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=181
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=182
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=183
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=184
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=185
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=186
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=187
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=188
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=189
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=190
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=191
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=192
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=193
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=194
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=195
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=196
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=197
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=198
DcpNozzle 2017-04-25T13:14:23.415-05:00 [INFO] There is no active stream for vb=199
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=259
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=260
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=261
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=262
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=263
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=264
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=265
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=266
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=267
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=268
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=269
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=270
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=271
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=272
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=273
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=274
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=275
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=276
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=277
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=278
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=279
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=280
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=281
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=282
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=283
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=284
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=285
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=286
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=287
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=288
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=289
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=290
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=291
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=292
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=293
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=294
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=295
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=296
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=297
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=298
DcpNozzle 2017-04-25T13:14:23.438-05:00 [INFO] There is no active stream for vb=299
DcpNozzle 2017-04-25T13:14:23.440-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:14:23.440-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:14:23.440726351 -0500 CDT
DcpNozzle 2017-04-25T13:14:23.440-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:14:23.440-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:14:23.440-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:14:23.442-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:14:23.443-05:00 [INFO] releasing capi client
DcpNozzle 2017-04-25T13:14:23.449-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:14:23.449-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:14:23.449872122 -0500 CDT
DcpNozzle 2017-04-25T13:14:23.449-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:14:23.449-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:14:23.449-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:14:23.449-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:14:23.450-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
ReplicationManager 2017-04-25T13:14:23.583-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
XDCRFactory 2017-04-25T13:14:23.591-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:14:23.597-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:14:23.597-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
PipelineManager 2017-04-25T13:14:23.690-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:14:12.538770796-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={2 source nozzles have been constructed}
]
XDCRFactory 2017-04-25T13:14:23.763-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:14:23.768-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:14:23.771-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:14:23.771-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:14:23.771-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:14:23.771-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:14:23.771-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:14:23.775-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:14:23.781-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
GenericSupervisor 2017-04-25T13:14:23.786-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:14:23.786-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
GenericSupervisor 2017-04-25T13:14:23.787-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:14:23.787-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:14:23.787-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:14:23.787-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:14:23.787-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:14:23.797-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
CheckpointManager 2017-04-25T13:14:23.797-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
AsyncListener 2017-04-25T13:14:23.877-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:14:23.877-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:14:23.877-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiUtils 2017-04-25T13:14:23.882-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:14:23.883-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:14:23.883-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:14:23.885-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
StatisticsManager 2017-04-25T13:14:23.893-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:14:23.894-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
XDCRFactory 2017-04-25T13:14:23.894-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:14:23.894-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 is constructed, start it
GenericPipeline 2017-04-25T13:14:23.897-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[260 293 199 265 266 269 205 222 231 259 191 214 215 279 281 306 340 217 223 181 221 294 173 180 261 309 308 317 288 297 216 213 228 319 243 247 290 203 258 339 251 333 336 183 210 270 271 289 208 201 291 292 194 220 253 283 302 335 192 241 284 320 188 177 193 234 329 331 175 312 326 248 324 232 171 256 272 249 264 239 296 321 187 282 311 200 278 207 229 323 300 322 274 276 225 298 262 277 307 318 198 195 252 303 338 185 172 182 197 219 246 257 299 227 301 316 212 226 237 238 275 184 268 230 250 174 176 179 242 204 189 325 196 328 287 240 330 263 314 315 235 254 304 218 178 209 236 280 190 233 255 285 286 186 206 332 202 267 327 337 245 310 224 244 295 313 334 211 273 305] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[30 32 116 14 58 104 5 72 146 150 165 20 63 130 18 108 121 2 10 161 129 74 164 38 122 149 162 66 80 102 96 155 12 52 90 133 163 48 68 81 86 111 158 16 94 97 45 124 134 1 25 29 140 35 93 138 160 60 157 37 131 153 8 65 92 54 79 15 19 34 151 136 53 148 4 43 159 88 23 40 114 143 78 115 46 67 77 55 125 145 169 47 83 71 117 127 56 119 106 168 6 61 95 9 27 39 147 170 22 59 107 110 17 84 105 112 24 7 91 76 49 57 73 36 82 11 70 100 156 167 51 139 152 13 109 120 31 87 103 62 26 113 42 132 135 50 128 154 28 41 101 141 89 98 126 44 99 118 137 3 64 85 69 75 123 142 144 166 0 21 33]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[86 111 158 16 94 97 45 124 134 1 25 29 140 35 93 138 160 60 157 37 131 153 8 65 92 54 79 15 19 34 151 136 53 148 4 43 159 88 23 40 114 143 78 115 46 67 77 55 125 145 169 47 83 71 117 127 56 119 106 168 6 61 95 9 27 39 147 170 22 59 107 110 17 84 105 112 24 7 91 76 49 57 73 36 82 11 70 100 156 167 51 139 152 13 109 120 31 87 103 62 26 113 42 132 135 50 128 154 28 41 101 141 89 98 126 44 99 118 137 3 64 85 69 75 123 142 144 166 0 21 33 30 32 116 14 58 104 5 72 146 150 165 20 63 130 18 108 121 2 10 161 129 74 164 38 122 149 162 66 80 102 96 155 12 52 90 133 163 48 68 81] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[258 339 251 333 336 183 210 270 271 289 208 201 291 292 194 220 253 283 302 335 192 241 284 320 188 177 193 234 329 331 175 312 326 248 324 232 171 256 272 249 264 239 296 321 187 282 311 200 278 207 229 323 300 322 274 276 225 298 262 277 307 318 198 195 252 303 338 185 172 182 197 219 246 257 299 227 301 316 212 226 237 238 275 184 268 230 250 174 176 179 242 204 189 325 196 328 287 240 330 263 314 315 235 254 304 218 178 209 236 280 190 233 255 285 286 186 206 332 202 267 327 337 245 310 224 244 295 313 334 211 273 305 260 293 199 265 266 269 205 222 231 259 191 214 215 279 281 306 340 217 223 181 221 294 173 180 261 309 308 317 288 297 216 213 228 319 243 247 290 203]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true failure_restart_interval:10 source_nozzle_per_node:2 doc_batch_size_kb:2048 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: checkpoint_interval:1800 worker_batch_size:500 stats_interval:1000]
CheckpointManager 2017-04-25T13:14:23.899-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055...
AsyncListener 2017-04-25T13:14:23.932-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:14:23.933-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:14:23.933-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:14:23.934-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:14:23.934-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:14:23.935-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:14:23.934-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:14:23.937-05:00 [INFO] Checkpointing starts in 1802.334 sec
PipelineRuntimeCtx 2017-04-25T13:14:23.937-05:00 [INFO] Service CheckpointManager has been started
CheckpointManager 2017-04-25T13:14:23.938-05:00 [INFO] checkpointing rountine started
StatisticsManager 2017-04-25T13:14:23.938-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:14:23.940-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:14:23.940-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:14:23.940-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:14:23.940-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:14:23.944-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:14:23.973-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:14:23.973-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:14:23.973-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:14:23.986-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:14:23.986-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:14:24.039-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:14:24.160-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:14:24.160-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:14:24.160-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
CapiUtils 2017-04-25T13:14:24.183-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:14:24.184-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:14:24.185-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:14:24.186-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:14:24.186-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:14:24.186-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:14:24.204-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:14:24.204-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:14:24.204-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:14:24.219-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:14:24.219-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:14:24.237-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:14:24.344-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:14:24.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:14:24.452-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:14:24.452-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:14:24.453-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:14:24.453-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:14:24.453-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:14:24.453-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 is started----------
PipelineManager 2017-04-25T13:14:24.453-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:14:24.453-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:14:24.466-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:14:24.616-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:14:26.618-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:14:26.532-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:14:26.480-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:14:26.565-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CAPIService 2017-04-25T13:15:05.110-05:00 [INFO] Received error when making rest call. baseURL=10.202.98.33:9091, path=_pre_replicate, ret_err=Post http://10.202.98.33:9091/_pre_replicate: EOF, statusCode=0, num_retry=5
GenericSupervisor 2017-04-25T13:15:05.120-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:15:05.059-05:00 [INFO] validateSourceTopology...
CAPIService 2017-04-25T13:15:05.105-05:00 [INFO] Received error when making rest call. baseURL=10.202.98.33:9091, path=_pre_replicate, ret_err=Post http://10.202.98.33:9091/_pre_replicate: EOF, statusCode=0, num_retry=5
StatisticsManager 2017-04-25T13:15:05.059-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:15:05 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:15:05.587-05:00 [INFO] Send batch count=23
StatisticsManager 2017-04-25T13:15:05.605-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:14:12.538770796-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 23, "expiry_received_from_dcp": 0, "set_received_from_dcp": 23}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:15:05.605-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:15:05.605-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 37 items, sent 23 items
StatisticsManager 2017-04-25T13:15:05.608-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:37]
StatisticsManager 2017-04-25T13:15:05.609-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 38 items, sent 37 items. streams inactive: [35 134 159 27 73 166 167 169 165 17 36 47 124 137 102 4 56 71 92 97 87 123 125 49 135 142 155 170 120 130 132 10 23 50 80 107 148 154 162 85 138 150 19 41 45 46 68 153 160 7 24 28 51 82 42 74 119 3 9 12 29 32 141 78 89 11 62 67 43 164 39 40 61 99 133 157 13 66 81 86 103 5 22 48 52 75 139 140 147 2 53 88 104 126 168 129 152 21 37 64 100 101 31 116 127 33 38 59 60 94 55 122 15 30 83 96 121 72 77 95 98 114 93 108 112 117 6 76 105 161 8 16 110 146 158 156 163 20 54 57 115 144 106 151 14 25 26 44 84 118 136 145 18 58 69 109 111 131 143 149 1 63 65 79 90 34 70 91 113 128]
StatisticsManager 2017-04-25T13:15:05.609-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:15:05.609-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [318 340 183 228 241 313 214 239 266 277 329 338 178 263 220 282 200 203 289 231 240 278 304 310 182 219 181 190 242 256 258 261 308 324 173 177 326 257 323 189 208 193 205 227 232 236 292 172 185 325 294 298 280 199 279 267 268 283 303 217 247 222 253 254 260 176 197 328 339 192 296 285 312 317 211 250 269 271 291 300 301 309 216 224 336 249 307 235 243 284 305 331 206 251 229 293 194 223 295 297 316 333 230 233 218 265 281 287 288 302 180 184 327 332 225 244 264 272 274 306 179 188 334 319 321 314 196 215 311 191 259 248 270 320 226 234 238 252 335 187 213 290 330 174 195 299 337 210 221 212 237 246 262 275 286 171 198 204 207 273 276 186 201 209 245 255 315 322 175 202]
StatisticsManager 2017-04-25T13:15:05.609-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =15 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =14 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:05.610-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
2017/04/25 13:15:05 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:15:05 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:15:05.917-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:05.945-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:15:06 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:15:06.066-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
GenericSupervisor 2017-04-25T13:15:06.085-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:15:06.087-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:15:06.087-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:15:06.385-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:15:06.446-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:15:06 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:15:06.490-05:00 [INFO] Send batch count=71
CapiNozzle 2017-04-25T13:15:06.588-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
PipelineManager 2017-04-25T13:15:06.609-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
StatisticsManager 2017-04-25T13:15:06.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:15:06 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 7 successful
CapiNozzle 2017-04-25T13:15:06.898-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:06.899-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:15:06 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:15:06 UPR_STREAMREQ for vb 109 successful
CapiNozzle 2017-04-25T13:15:07.043-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:15:07 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:15:07 UPR_STREAMREQ for vb 110 successful
CapiNozzle 2017-04-25T13:15:07.084-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:15:07 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:15:07 UPR_STREAMREQ for vb 304 successful
CapiNozzle 2017-04-25T13:15:07.191-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:15:07 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:15:07 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:15:07 UPR_STREAMREQ for vb 13 successful
CapiNozzle 2017-04-25T13:15:07.921-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:15:08.438-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:15:08.438-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:15:08.438-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:15:08.439-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:15:08.442-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:15:08 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:15:08 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:15:08 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:15:08 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 114 successful
CapiNozzle 2017-04-25T13:15:09.060-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:15:09 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 207 successful
PipelineManager 2017-04-25T13:15:09.110-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:15:09.112-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:15:09.126-05:00 [INFO] err_list=[{"time":"2017-04-25T13:15:09.126995147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:15:09.140-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:15:09 UPR_STREAMREQ for vb 17 successful
StatisticsManager 2017-04-25T13:15:09.183-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:15:09.183-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:15:09.187-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 {"Errors": "[{\"time\":\"2017-04-25T13:15:09.126995147-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 263, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 41, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 3196, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 41, "set_filtered": 0, "set_received_from_dcp": 3196, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

2017/04/25 13:15:09 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:15:09 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 121 successful
CapiNozzle 2017-04-25T13:15:10.821-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:15:10 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:15:10 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:15:11.135-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:15:11 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 317 successful
CapiNozzle 2017-04-25T13:15:11.664-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:15:11 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 318 successful
CapiNozzle 2017-04-25T13:15:11.710-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:15:11 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:15:11 UPR_STREAMREQ for vb 126 successful
CapiNozzle 2017-04-25T13:15:11.950-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:15:11.970-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:15:11.970-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:15:11.970-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:15:11.970-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:11.988-05:00 [INFO] Send batch count=465
ReplicationManager 2017-04-25T13:15:12.088-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:15:12 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 135 successful
CapiNozzle 2017-04-25T13:15:12.560-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:12.560-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:15:12 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 40 successful
CapiNozzle 2017-04-25T13:15:12.679-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
DcpNozzle 2017-04-25T13:15:12.698-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
2017/04/25 13:15:12 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 41 successful
PipelineManager 2017-04-25T13:15:12.833-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:09.126995147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
2017/04/25 13:15:12 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:15:12 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 237 successful
CapiNozzle 2017-04-25T13:15:13.039-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:15:13.063-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:15:13 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 47 successful
CapiNozzle 2017-04-25T13:15:13.259-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:15:13.281-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:15:13 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 149 successful
CapiNozzle 2017-04-25T13:15:13.670-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:15:13 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 248 successful
CapiNozzle 2017-04-25T13:15:13.723-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:15:13 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 64 successful
GenericSupervisor 2017-04-25T13:15:13.951-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:15:13.954-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:15:13.957-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:15:13 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:15:13 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 260 successful
CapiNozzle 2017-04-25T13:15:14.221-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:15:14 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 164 successful
CapiNozzle 2017-04-25T13:15:14.294-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:15:14 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 266 successful
ToplogyChangeDetector 2017-04-25T13:15:14.437-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:15:14 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 176 successful
CapiNozzle 2017-04-25T13:15:14.789-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:15:14.789-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:15:14.789-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:15:14.789-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:14.789-05:00 [INFO] Send batch count=500
2017/04/25 13:15:14 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:15:14 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 188 successful
CapiNozzle 2017-04-25T13:15:15.320-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:15:15.320-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:15:15.320-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:15:15.320-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:15.320-05:00 [INFO] Send batch count=500
2017/04/25 13:15:15 UPR_STREAMREQ for vb 98 successful
CapiNozzle 2017-04-25T13:15:15.379-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:15.383-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:15:15 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 291 successful
CapiNozzle 2017-04-25T13:15:15.444-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:15:15 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 191 successful
DcpNozzle 2017-04-25T13:15:15.544-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:15:15.577-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:15.577-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:15.597-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:15:15 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 294 successful
CheckpointManager 2017-04-25T13:15:15.684-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
CheckpointManager 2017-04-25T13:15:15.684-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:15:15 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:15:15 UPR_STREAMREQ for vb 198 successful
DcpNozzle 2017-04-25T13:15:15.855-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:15:16.030-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:15:16.049-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:15:16.419-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:16.507-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:15:17.569-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:15:17.612-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
PipelineManager 2017-04-25T13:15:19.294-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
CapiNozzle 2017-04-25T13:15:19.591-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
PipelineManager 2017-04-25T13:15:19.667-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:15:19.672-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:15:19.675-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
PipelineRuntimeCtx 2017-04-25T13:15:19.815-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:15:19.824-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:15:19.830-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:15:19.852-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:19.829623275 -0500 CDT
GenericSupervisor 2017-04-25T13:15:19.985-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:15:19.985-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:15:19.987-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:15:19.989-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:15:19.990-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:15:19.990-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:15:19.991-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:15:19.992-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:15:19.994-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:15:19.994-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 has been stopped
CapiNozzle 2017-04-25T13:15:20.122-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
GenericPipeline 2017-04-25T13:15:20.433-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-771871055 is stopped
PipelineManager 2017-04-25T13:15:20.434-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:15:20.436-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:09.126995147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:15:20.444-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:15:20.454-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:20.459-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:20.459-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:20.459-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:20.459-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.459-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.461-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.461-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.461-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:15:20.436-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
AsyncListener 2017-04-25T13:15:20.438-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:15:20.437-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
GenericPipeline 2017-04-25T13:15:20.489-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:15:20.490-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:15:20.500-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:15:20.508-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:15:20.437-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
GenericPipeline 2017-04-25T13:15:20.508-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:15:20.509-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:15:20.514-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:15:20.437-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-771871055
GenericPipeline 2017-04-25T13:15:20.515-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:15:20.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:15:20.515-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:15:20.516-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:20.514527171 -0500 CDT
GenericPipeline 2017-04-25T13:15:20.517-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:15:20.517-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:15:20.517-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:15:20.517-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:15:20.519-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:20.519-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:15:20.519-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:20.517975861 -0500 CDT
CapiNozzle 2017-04-25T13:15:20.519-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
DcpNozzle 2017-04-25T13:15:20.555-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:15:20.555-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:20.555851406 -0500 CDT
DcpNozzle 2017-04-25T13:15:20.556-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:15:20.556-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:15:20.556-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:15:20.556-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:20.559-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:15:20.631-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
DcpNozzle 2017-04-25T13:15:20.639-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:15:20.639-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:20.639842353 -0500 CDT
CapiNozzle 2017-04-25T13:15:20.640-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:15:20.640-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:20.640-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:15:20.639-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:15:20.641-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:20.641-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:15:21.041-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:15:21.053-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:15:21.054-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
CapiNozzle 2017-04-25T13:15:21.328-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:15:21.391-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:15:21.406-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:21.456-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:15:21.816-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:15:24.519-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:15:24.655-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:15:24.655-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:15:24.655-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:15:24.655-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:15:24.658-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:15:24.756-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:15:24.841-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
GenericSupervisor 2017-04-25T13:15:24.863-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:15:24.865-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
GenericSupervisor 2017-04-25T13:15:24.866-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:24.866-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:24.866-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:24.866-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:24.866-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:15:24.877-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
CheckpointManager 2017-04-25T13:15:24.877-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
PipelineManager 2017-04-25T13:15:25.011-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:09.126995147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Source nozzles are wired to target nozzles}
]
CapiUtils 2017-04-25T13:15:25.088-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:15:25.090-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:15:25.091-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:15:25.109-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
StatisticsManager 2017-04-25T13:15:25.140-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:15:25.142-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
XDCRFactory 2017-04-25T13:15:25.144-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:15:25.148-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 is constructed, start it
GenericPipeline 2017-04-25T13:15:25.161-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[44 113 31 158 81 86 167 62 82 21 163 14 7 53 57 140 61 138 132 107 161 73 16 98 11 162 149 128 18 19 126 127 20 79 145 0 26 78 23 123 165 106 80 148 33 76 139 36 72 89 119 124 125 170 52 48 46 112 147 169 42 108 66 55 63 90 117 54 118 84 91 38 137 50 166 25 101 103 4 40 51 3 116 87 168 109 92 104 105 129 45 96 111 114 153 24 77 9 27 120 8 155 136 22 121 17 83 35 135 157 2 144 164 56 156 68 30 134 15 64 122 1 142 47 29 59 93 32 141 12 10 85 100 102 110 115 6 95 159 69 97 133 160 5 49 67 146 151 37 143 71 74 150 58 43 60 94 28 75 88 65 154 70 39 34 41 99 130 131 152 13] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[179 208 226 313 319 330 218 257 266 217 237 245 281 177 276 317 323 243 262 277 295 258 292 301 328 339 185 211 173 183 271 310 334 250 287 187 285 318 291 188 216 219 273 335 278 290 272 294 280 213 255 223 274 338 202 222 229 244 325 340 190 297 192 282 298 308 311 326 175 212 178 260 247 300 227 259 329 203 214 337 205 225 307 201 238 322 186 171 224 331 215 261 324 207 253 194 206 233 312 172 234 235 204 230 314 240 275 309 321 174 198 242 264 269 288 265 200 241 254 305 293 302 221 191 239 279 299 195 320 193 256 333 248 284 252 270 315 181 220 268 286 304 196 283 182 327 180 197 246 332 306 228 289 184 199 232 267 336 231 296 303 189 209 236 249 210 316 176 263 251]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[35 83 2 135 157 56 144 164 68 156 15 30 134 1 64 122 47 142 29 32 59 93 12 141 110 115 6 10 85 100 102 69 95 159 5 97 133 160 37 49 67 146 151 143 58 71 74 150 28 43 60 94 65 75 88 70 154 39 131 152 13 34 41 99 130 31 44 113 81 158 62 86 167 82 14 21 163 7 53 57 61 140 132 138 73 107 161 11 16 98 149 162 18 128 19 20 126 127 0 79 145 23 26 78 106 123 165 33 80 148 36 76 139 125 170 52 72 89 119 124 46 48 42 112 147 169 66 108 54 55 63 90 117 118 38 84 91 50 137 166 4 25 101 103 3 40 51 87 116 109 168 45 92 104 105 129 96 24 111 114 153 9 77 8 27 120 136 155 17 22 121] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[200 241 254 305 293 302 221 279 299 191 239 195 320 193 256 333 248 284 252 270 315 181 196 220 268 286 304 283 327 182 180 197 246 306 332 228 289 199 232 267 184 336 303 231 296 249 189 209 236 210 316 176 251 263 226 313 319 179 208 330 218 257 266 245 281 217 237 177 323 276 317 243 262 277 295 339 258 292 301 328 185 211 310 334 173 183 271 250 287 318 187 285 291 273 335 188 216 219 278 290 272 294 280 213 255 338 223 274 229 244 325 202 222 340 190 297 308 311 192 282 298 326 175 212 178 260 247 300 227 259 329 203 214 337 201 205 225 307 322 238 186 171 224 331 215 261 324 207 253 206 233 312 194 172 234 235 314 204 230 321 240 275 309 242 264 269 174 198 288 265]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 replication_type:capi active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 stats_interval:1000 filter_expression: worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:15:25.171-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774...
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:25.214-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:15:25.216-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:15:25.217-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:15:25.217-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:15:25.218-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:15:25.227-05:00 [INFO] Checkpointing starts in 1804.908 sec
PipelineRuntimeCtx 2017-04-25T13:15:25.227-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:15:25.228-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:15:25.231-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:15:25.242-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:15:25.262-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:15:25.266-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:15:25.266-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:15:25.273-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:15:25.289-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:15:25.387-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:15:25.387-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:15:25.395-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:15:25.442-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:15:25.524-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:15:25.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:15:25.663-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:15:29.249-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:15:29.269-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:15:29.234-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:15:29.208-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
StatisticsManager 2017-04-25T13:15:29.827-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:30.271-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:30.707-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:31.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:32.737-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:34.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:35.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:35.418-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:15:09.126995147-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:15:35.423-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:15:35.423-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:15:35.431-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:15:35.438-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:15:35.438-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:15:35.438-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:15:35.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
GenericSupervisor 2017-04-25T13:15:35.473-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:15:35.912-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:15:36.423-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:15:36.587-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:15:36.589-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:15:36.589-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:15:38.730-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:15:39.000-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:15:39.424-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:09.126995147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={The runtime context is started}
]
StatisticsManager 2017-04-25T13:15:39.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:15:39.873-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:15:39.887-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:15:39.888-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:15:39.893-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:15:39.895-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:15:39.992-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:15:39.993-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:15:39.993-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:15:40.002-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:15:40.003-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:15:40.006-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:15:40.176-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:15:40.176-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:15:40.176-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:15:40.188-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:15:40.188-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:15:40.334-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:15:40.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:15:40.365-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:15:40.365-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:15:40.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:15:40.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:15:40.366-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:15:40.368-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 is started----------
PipelineManager 2017-04-25T13:15:40.372-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:15:40.372-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:15:40.373-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:15:40 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:15:40 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 227 successful
StatisticsManager 2017-04-25T13:15:41.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:15:41 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 30 successful
CapiNozzle 2017-04-25T13:15:41.368-05:00 [INFO] Send batch count=116
2017/04/25 13:15:41 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:15:41.597-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:15:41.601-05:00 [INFO] Send batch count=54
2017/04/25 13:15:41 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 281 successful
CapiNozzle 2017-04-25T13:15:41.776-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:41.790-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:15:41 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 81 successful
CapiNozzle 2017-04-25T13:15:41.874-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:15:41.897-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
2017/04/25 13:15:41 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:15:41 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 193 successful
CheckpointManager 2017-04-25T13:15:42.132-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
CheckpointManager 2017-04-25T13:15:42.132-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:15:42 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:15:42 UPR_STREAMREQ for vb 196 successful
DcpNozzle 2017-04-25T13:15:42.207-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:15:42.221-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:42.235-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:15:42.267-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
DcpNozzle 2017-04-25T13:15:42.268-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:15:42.308-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
StatisticsManager 2017-04-25T13:15:42.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=5, changes_left=315503
CapiNozzle 2017-04-25T13:15:42.636-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:42.656-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:15:42.684-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:15:42.699-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:15:42.840-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:42.840-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:15:42.840-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:15:42.840-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:15:42.842-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:15:42.865-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:15:42.865-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:15:42.871-05:00 [INFO] err_list=[{"time":"2017-04-25T13:15:42.871093417-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:15:42.876-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:15:42.891-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:15:42.891-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:15:42.891-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:15:42.891-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:42.891-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:42.954-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:42.960-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:42.985-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:15:42.988-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:42.988-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:43.008-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
StatisticsManager 2017-04-25T13:15:43.262-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:15:43.263-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 {"Errors": "[{\"time\":\"2017-04-25T13:15:42.871093417-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315503, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 68, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 5, "docs_received_from_dcp": 19472, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 68, "set_filtered": 0, "set_received_from_dcp": 19472, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:15:43.434-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:43.447-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:15:43.637-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:15:43.647-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:15:44.177-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:44.191-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:15:44.421-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:15:44.432-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:15:44.649-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:15:44.649-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:15:44.649-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:15:44.649-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:44.649-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:44.748-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:44.748-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:44.765-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
GenericSupervisor 2017-04-25T13:15:45.239-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:15:45.239-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:15:45.239-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:15:45.257-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:15:45.257-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:15:45.257-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:15:45.257-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:45.257-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:45.356-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:45.356-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:45.377-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:15:45.476-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:45.500-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:15:46.073-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:15:46.108-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:15:46.259-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:46.281-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:15:46.903-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:15:46.927-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:15:47.214-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:47.214-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:15:47.214-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:15:47.214-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:47.215-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:47.323-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:47.323-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:47.338-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:15:47.908-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:15:47.908-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:15:47.908-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:15:47.908-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:47.908-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:48.040-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:48.040-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:48.133-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:15:48.941-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:15:48.967-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:15:49.305-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:15:49.327-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:15:49.643-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:15:49.665-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:15:50.209-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:15:50.363-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:15:50.366-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:15:50.366-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:15:50.368-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:50.417-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:50.635-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:15:50.994-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:51.165-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:51.193-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:15:51.707-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:15:51.816-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:15:51.818-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:15:51.818-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:15:51.825-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:15:51.881-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:15:52.015-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
PipelineManager 2017-04-25T13:15:52.985-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:15:54.098-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:15:54.142-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:15:54.211-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
PipelineRuntimeCtx 2017-04-25T13:15:54.263-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:15:54.391-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:15:54.409-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:15:54.515-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:54.395419052 -0500 CDT
GenericSupervisor 2017-04-25T13:15:54.634-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:15:54.634-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:15:54.649-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:15:54.658-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:15:54.664-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:15:54.664-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:15:54.668-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:15:54.681-05:00 [INFO] StatisticsManager Stopped
CapiNozzle 2017-04-25T13:15:54.731-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
ToplogyChangeDetector 2017-04-25T13:15:54.734-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:15:54.734-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 has been stopped
GenericPipeline 2017-04-25T13:15:54.995-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-656345774 is stopped
PipelineManager 2017-04-25T13:15:54.995-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:15:55.159-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
PipelineManager 2017-04-25T13:15:55.000-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:42.871093417-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

AsyncListener 2017-04-25T13:15:55.180-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:15:55.200-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:15:55.104-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
CapiNozzle 2017-04-25T13:15:54.979-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
PipelineManager 2017-04-25T13:15:55.248-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:15:55.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:15:55.115-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
GenericPipeline 2017-04-25T13:15:55.261-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:15:55.272-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
CapiNozzle 2017-04-25T13:15:55.299-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
AsyncListener 2017-04-25T13:15:55.299-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:15:55.115-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
GenericPipeline 2017-04-25T13:15:55.299-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:15:55.299-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:15:55.301-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:15:55.312-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:15:55.115-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-656345774
GenericPipeline 2017-04-25T13:15:55.312-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:15:55.312-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:15:55.327-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.327-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:15:55.333-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:55.312756156 -0500 CDT
GenericPipeline 2017-04-25T13:15:55.333-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:15:55.333-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:15:55.333-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:15:55.333-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:15:55.333-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:15:55.333-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:15:55.333-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:15:55.333-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:55.333560713 -0500 CDT
CapiNozzle 2017-04-25T13:15:55.333-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
DcpNozzle 2017-04-25T13:15:55.333-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:15:55.401-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:15:55.312-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:15:55.415-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:15:55.458-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:15:55.578-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:15:55.578-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:15:55.578-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:55.580-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:15:55.582-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:15:55.583-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:55.582962388 -0500 CDT
DcpNozzle 2017-04-25T13:15:55.583-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:15:55.583-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:55.583-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:15:55.585-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:15:55.585-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:15:55.585237851 -0500 CDT
DcpNozzle 2017-04-25T13:15:55.585-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:15:55.585-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:55.585-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:15:55.675-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:15:55.676-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:15:55.676-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:15:55.676-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
PipelineManager 2017-04-25T13:15:56.476-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:42.871093417-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:15:56.627-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:15:56.631-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:15:56.631-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:15:56.698-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:15:56.702-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:15:56.705-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:15:56.705-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:15:56.705-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:15:56.705-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:15:56.705-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:15:56.707-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:15:56.710-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
GenericSupervisor 2017-04-25T13:15:56.717-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:15:56.717-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
GenericSupervisor 2017-04-25T13:15:56.718-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:56.718-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:56.718-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:56.718-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:15:56.718-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:15:56.722-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
CheckpointManager 2017-04-25T13:15:56.722-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
CapiUtils 2017-04-25T13:15:56.752-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:15:56.752-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:15:56.753-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:15:56.759-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
StatisticsManager 2017-04-25T13:15:56.767-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:15:56.768-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
XDCRFactory 2017-04-25T13:15:56.768-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:15:56.769-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799 is constructed, start it
GenericPipeline 2017-04-25T13:15:56.771-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[177 233 251 257 269 334 197 228 255 259 226 229 254 196 201 241 256 266 212 314 293 298 249 300 299 216 188 301 328 172 175 176 200 248 183 213 243 244 278 282 316 194 195 191 193 208 279 286 296 308 309 317 339 209 306 320 332 203 218 260 272 288 207 235 322 204 206 227 174 198 246 221 295 304 274 324 326 331 214 330 171 273 287 312 333 262 291 261 280 285 297 192 276 284 290 283 179 180 238 305 234 277 187 215 217 302 340 236 313 173 222 270 223 224 303 315 281 210 307 323 237 258 275 178 181 319 247 336 253 250 264 267 220 242 318 205 232 252 338 182 184 310 199 219 294 327 185 321 190 230 335 292 240 263 311 337 239 325 186 202 211 231 329 268 271 189 225 245 265 289] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[19 40 93 29 110 74 129 58 158 43 16 21 44 63 75 102 140 144 80 52 69 76 160 169 84 137 155 164 65 98 130 5 30 34 49 131 50 101 115 135 0 18 38 154 54 95 142 64 66 79 147 149 22 37 68 11 106 162 41 56 90 7 104 28 165 70 138 156 107 168 109 86 99 112 141 51 105 1 20 42 118 134 2 73 161 48 108 6 89 146 59 85 36 67 88 122 123 132 14 157 45 91 10 60 114 128 71 121 151 152 159 167 57 81 13 32 26 92 97 143 25 27 61 113 163 8 53 116 120 15 82 62 124 24 139 153 9 55 78 117 47 46 103 87 83 133 12 111 4 39 136 148 23 150 72 125 31 77 94 100 145 3 35 96 119 126 17 33 166 170 127]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[38 154 18 95 142 54 66 79 147 149 64 37 68 22 106 162 11 56 90 41 104 7 165 28 138 156 70 168 107 109 99 112 141 86 105 51 20 42 118 134 1 73 161 2 108 48 89 146 6 85 59 67 88 122 123 132 36 157 14 91 45 60 114 128 10 121 151 152 159 167 71 57 81 32 13 92 97 26 143 27 61 113 163 25 53 116 120 8 82 15 124 62 139 153 24 55 78 117 9 47 103 46 87 133 83 111 12 39 136 148 4 150 23 125 72 77 94 100 145 31 35 96 119 126 3 17 166 170 33 127 40 93 19 110 29 129 74 158 58 43 21 44 63 75 102 140 144 16 80 69 76 160 169 52 137 155 164 84 98 130 65 30 34 49 131 5 101 115 135 50 0] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[194 195 191 193 208 279 286 308 296 309 317 339 209 306 320 332 203 218 260 272 288 207 235 322 204 206 227 174 198 246 221 295 304 274 324 326 331 214 330 171 273 287 312 333 262 291 261 280 285 297 192 276 284 290 283 179 180 238 305 277 234 187 215 217 302 340 236 313 173 222 270 223 224 303 315 281 210 307 323 237 258 275 178 181 319 247 336 253 250 264 267 220 242 318 205 232 252 338 182 184 310 199 219 294 327 185 321 190 230 335 292 240 263 311 337 239 325 186 202 211 231 329 268 271 189 225 245 265 289 233 251 257 269 334 177 197 228 255 259 226 229 254 196 201 241 256 266 212 314 298 293 249 300 299 216 188 301 328 172 175 176 248 200 183 213 243 244 278 282 316]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: worker_batch_size:500 log_level:Info doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi active:true checkpoint_interval:1800 stats_interval:1000]
CheckpointManager 2017-04-25T13:15:56.773-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799...
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:15:56.780-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
CheckpointManager 2017-04-25T13:15:56.782-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:15:56.786-05:00 [INFO] Checkpointing starts in 1804.899 sec
PipelineRuntimeCtx 2017-04-25T13:15:56.786-05:00 [INFO] Service CheckpointManager has been started
CheckpointManager 2017-04-25T13:15:56.786-05:00 [INFO] checkpointing rountine started
StatisticsManager 2017-04-25T13:15:56.787-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:15:56.797-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:15:56.797-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:15:56.798-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:15:56.798-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:15:56.798-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:15:56.799-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:15:56.799-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:15:56.802-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:15:56.805-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:15:56.825-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:15:56.825-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:15:56.825-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:15:56.836-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:15:56.846-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:15:56.846-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:15:56.859-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:15:56.859-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:15:56.860-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:15:56.870-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:15:56.916-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:15:56.936-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:15:56.936-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:15:56.936-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:15:56.940-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:15:56.941-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:15:56.941-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:15:56.956-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:15:56.956-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:15:56.956-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:15:56.981-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:15:56.981-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:15:57.028-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:15:57.147-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:15:57.147-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:15:57.147-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:15:57.147-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:15:57.147-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:15:57.147-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:15:57.147-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799 is started----------
PipelineManager 2017-04-25T13:15:57.148-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:15:57.148-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:15:57.155-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:15:57.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:15:58.035-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:15:58.040-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:15:58.040-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:15:58.040-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:15:58 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:15:58.429-05:00 [INFO] Send batch count=95
2017/04/25 13:15:58 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 107 successful
CapiNozzle 2017-04-25T13:15:58.452-05:00 [INFO] Send batch count=215
2017/04/25 13:15:58 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 224 successful
CapiNozzle 2017-04-25T13:15:58.564-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:58.582-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:15:58 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 256 successful
CapiNozzle 2017-04-25T13:15:58.876-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:15:58.889-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
StatisticsManager 2017-04-25T13:15:58.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:15:58 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:15:58 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 178 successful
CapiNozzle 2017-04-25T13:15:59.196-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:59.206-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:15:59 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 287 successful
CheckpointManager 2017-04-25T13:15:59.289-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
CheckpointManager 2017-04-25T13:15:59.289-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:15:59 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:15:59 UPR_STREAMREQ for vb 199 successful
CapiNozzle 2017-04-25T13:15:59.415-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:15:59.426-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
DcpNozzle 2017-04-25T13:15:59.442-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:15:59.447-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:15:59.463-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:59.474-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:15:59.764-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:15:59.764-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:15:59.764-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:15:59.764-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]
ReplicationManager 2017-04-25T13:15:59.765-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]
PipelineManager 2017-04-25T13:15:59.780-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:15:59.781-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:15:59.783-05:00 [INFO] err_list=[{"time":"2017-04-25T13:15:59.78336066-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]
PipelineManager 2017-04-25T13:15:59.790-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:15:59.799-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:15:59.799-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799 {"Errors": "[{\"time\":\"2017-04-25T13:15:59.78336066-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 140, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 11177, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 140, "set_filtered": 0, "set_received_from_dcp": 11177, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:15:59.859-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:15:59.860-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:15:59.870-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:15:59.988-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:15:59.999-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:16:00.512-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:16:00.526-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:16:00.682-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:16:00.682-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:16:00.682-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:16:00.682-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:16:00.682-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:16:00.753-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:00.753-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:16:00.765-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:16:01.187-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:16:01.204-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:16:01.370-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:16:01.389-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:16:02.325-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:16:02.426-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:16:03.909-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:16:03.529-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:16:04.190-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:16:05.739-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:16:05.429-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:16:06.874-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:16:07.053-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:16:07.000-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:16:07.074-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:16:07.234-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:07.323-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:16:07.480-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:16:07.485-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:16:07.485-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:16:07.485-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:16:07.492-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:16:07.669-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:16:07.781-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:07.781-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:16:07.806-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
PipelineManager 2017-04-25T13:16:08.128-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:59.78336066-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]}
]
CapiNozzle 2017-04-25T13:16:08.219-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:08.236-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:16:08.660-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:16:08.677-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
ReplicationManager 2017-04-25T13:16:08.714-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:16:09.230-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:09.280-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:16:09.657-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:16:09.678-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
PipelineManager 2017-04-25T13:16:09.795-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries 
PipelineManager 2017-04-25T13:16:09.803-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:16:09.803-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:16:09.805-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
PipelineRuntimeCtx 2017-04-25T13:16:09.811-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:16:09.821-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:16:09.821-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:16:09.821-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:16:09.827-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:16:09.827-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:16:09.835-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:09.827909044 -0500 CDT
GenericSupervisor 2017-04-25T13:16:09.865-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:16:09.865-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:16:09.876-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:16:09.876-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:16:09.883-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:16:09.883-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:16:09.883-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-705424799 has been stopped
GenericPipeline 2017-04-25T13:16:09.906-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-705424799 is stopped
PipelineManager 2017-04-25T13:16:09.906-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:16:09.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:09.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:09.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:09.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.920-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:09.915-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
PipelineManager 2017-04-25T13:16:09.923-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:15:59.78336066-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:16:09.927-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:16:09.932-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
GenericPipeline 2017-04-25T13:16:09.934-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
GenericPipeline 2017-04-25T13:16:09.934-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
GenericPipeline 2017-04-25T13:16:09.934-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-705424799
AsyncListener 2017-04-25T13:16:09.934-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:09.934-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:09.934-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:09.934-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:16:09.934-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:16:09.935-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:16:09.938-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:16:09.940-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:16:09.940-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:16:09.948-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:16:09.948-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:16:09.948-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:16:09.948-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:16:09.948-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:16:09.948-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:16:09.948-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:16:09.948-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.948-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:09.948-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:16:09.948-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:16:09.948-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:09.948163618 -0500 CDT
DcpNozzle 2017-04-25T13:16:09.948-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:16:09.948-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:09.948480236 -0500 CDT
CapiNozzle 2017-04-25T13:16:09.949-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:16:09.949-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:16:09.949-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:16:09.949-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:16:09.949-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:16:09.952-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:16:09.952-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:09.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:16:09.955-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:09.956-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:16:09.956-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:09.956-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:16:09.962-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:16:09.962-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:09.962462503 -0500 CDT
DcpNozzle 2017-04-25T13:16:09.962-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:16:09.962-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:09.962-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:16:09.962-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:16:09.962-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:09.962840807 -0500 CDT
DcpNozzle 2017-04-25T13:16:09.962-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:16:09.962-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:09.962-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
XDCRFactory 2017-04-25T13:16:10.058-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:16:10.062-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:16:10.062-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:16:10.162-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:16:10.165-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:16:10.168-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:16:10.168-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:16:10.168-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:16:10.168-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:16:10.168-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:16:10.169-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:16:10.175-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
GenericSupervisor 2017-04-25T13:16:10.179-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:16:10.179-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
GenericSupervisor 2017-04-25T13:16:10.179-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:10.179-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:10.179-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:10.179-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:10.179-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:16:10.183-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
CheckpointManager 2017-04-25T13:16:10.183-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
CapiUtils 2017-04-25T13:16:10.261-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:16:10.261-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:16:10.262-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:16:10.264-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
StatisticsManager 2017-04-25T13:16:10.271-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:16:10.272-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
XDCRFactory 2017-04-25T13:16:10.272-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:16:10.272-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998 is constructed, start it
GenericPipeline 2017-04-25T13:16:10.273-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[40 122 21 34 19 52 87 133 156 51 43 75 147 32 1 103 154 5 107 111 121 141 92 129 82 114 45 137 131 48 109 2 66 76 120 148 64 60 118 27 50 132 170 26 57 59 108 6 157 86 161 159 16 47 124 12 38 123 10 54 167 0 99 138 11 125 79 158 166 97 93 116 127 88 3 136 140 155 162 126 33 152 15 144 145 31 130 95 105 7 25 41 42 163 13 37 56 80 91 119 149 153 9 36 115 135 71 89 46 164 65 72 151 165 20 83 112 134 146 67 168 22 61 68 117 14 85 102 106 35 58 53 70 128 150 69 23 44 8 63 74 77 142 39 100 81 96 110 139 143 29 84 98 49 28 30 55 94 104 113 4 78 101 160 17 73 62 24 90 18 169] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[201 213 216 227 259 293 242 246 249 270 187 218 315 208 233 277 284 302 323 316 245 287 231 266 273 180 243 324 196 238 265 281 322 239 279 320 179 190 225 292 193 210 228 261 263 296 185 275 271 221 257 321 319 200 223 256 178 332 337 338 189 310 171 212 177 255 188 260 286 298 307 325 198 276 174 252 267 274 224 240 295 313 195 222 306 285 215 264 269 289 176 203 220 280 262 311 172 294 330 317 173 206 300 318 250 299 340 278 301 304 272 182 253 314 194 197 209 226 339 184 244 282 305 235 205 219 237 309 202 232 290 297 303 291 207 214 254 312 335 308 186 234 204 230 247 251 327 328 334 181 192 336 236 331 329 191 199 211 248 268 326 183 241 283 288 175 217 229 258 333]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[21 40 122 19 34 51 52 87 133 156 32 43 75 147 1 5 103 154 92 107 111 121 141 82 129 45 114 131 137 2 48 109 66 76 64 120 148 27 60 118 26 50 132 170 6 57 59 108 86 157 161 16 159 12 47 124 10 38 123 0 54 167 11 99 138 79 125 97 158 166 93 88 116 127 3 126 136 140 155 162 15 33 152 31 144 145 130 7 95 105 13 25 41 42 163 9 37 56 80 91 119 149 153 36 71 115 135 46 89 65 164 20 72 151 165 67 83 112 134 146 168 22 14 61 68 117 35 85 102 106 53 58 69 70 128 150 8 23 44 39 63 74 77 142 81 100 96 29 110 139 143 49 84 98 4 28 30 55 94 104 113 17 78 101 160 62 73 18 24 90 169] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[201 213 216 227 259 293 242 246 249 270 187 218 315 208 233 277 284 302 323 316 231 245 287 266 273 180 243 324 196 238 265 281 322 239 279 320 179 190 225 292 193 210 228 261 263 296 185 275 271 221 257 321 319 200 223 256 178 332 337 338 189 310 171 212 177 255 188 260 286 298 307 325 198 276 174 252 267 274 224 240 295 313 195 222 306 285 215 264 269 289 176 203 220 262 280 311 172 294 330 317 173 206 300 318 250 299 340 272 278 301 304 182 253 314 194 197 209 226 339 184 244 282 305 235 205 219 237 309 202 232 290 297 303 291 207 214 254 308 312 335 186 234 204 230 247 251 327 328 334 181 192 336 236 329 331 191 199 211 248 268 326 183 241 283 288 175 217 229 258 333]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 stats_interval:1000 optimistic_replication_threshold:256 replication_type:capi filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:16:10.275-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998...
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:10.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:16:10.297-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:16:10.298-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:16:10.298-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:16:10.299-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:16:10.299-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:16:10.303-05:00 [INFO] Checkpointing starts in 1804.068 sec
PipelineRuntimeCtx 2017-04-25T13:16:10.303-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:16:10.303-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:16:10.303-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:16:10.304-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:16:10.305-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:16:10.306-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:16:10.306-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:16:10.311-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:16:10.330-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:16:10.331-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:16:10.331-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:16:10.336-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:16:10.336-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:16:10.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:16:10.382-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:16:10.399-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:16:10.399-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:16:10.401-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:16:10.418-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:16:10.421-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:16:10.421-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:16:10.421-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:16:10.426-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:16:10.427-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:16:10.428-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:16:10.445-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:16:10.445-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:16:10.445-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:16:10.464-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:16:10.464-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:16:10.529-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:16:10.619-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:16:10.619-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:16:10.619-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:16:10.619-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:16:10.619-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:16:10.620-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:16:10.620-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998 is started----------
PipelineManager 2017-04-25T13:16:10.620-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:16:10.620-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:16:10.621-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:16:11.438-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:16:11.488-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:16:11.497-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:16:11.499-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:16:11.500-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:16:11 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 6 successful
CapiNozzle 2017-04-25T13:16:11.920-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:16:11.925-05:00 [INFO] Send batch count=32
2017/04/25 13:16:11 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 310 successful
CapiNozzle 2017-04-25T13:16:11.932-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:11.932-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:16:11 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:16:11 UPR_STREAMREQ for vb 110 successful
CapiNozzle 2017-04-25T13:16:11.952-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:16:12 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 119 successful
CapiNozzle 2017-04-25T13:16:12.070-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:12.093-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:16:12 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 335 successful
CapiNozzle 2017-04-25T13:16:12.139-05:00 [INFO] Send batch count=280
2017/04/25 13:16:12 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 233 successful
CapiNozzle 2017-04-25T13:16:12.143-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:12.158-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:16:12.216-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:12.216-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:16:12 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 249 successful
CapiNozzle 2017-04-25T13:16:12.227-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:16:12 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 61 successful
CapiNozzle 2017-04-25T13:16:12.309-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:12.309-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:16:12.309-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:16:12.309-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
ReplicationManager 2017-04-25T13:16:12.312-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
PipelineManager 2017-04-25T13:16:12.321-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:16:12.321-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:16:12.324-05:00 [INFO] err_list=[{"time":"2017-04-25T13:16:12.324151353-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:16:12.336-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:16:12 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 65 successful
StatisticsManager 2017-04-25T13:16:12.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:16:12 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 169 successful
CapiNozzle 2017-04-25T13:16:12.447-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:12.447-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:16:12.460-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:16:12 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:16:12 UPR_STREAMREQ for vb 72 successful
StatisticsManager 2017-04-25T13:16:13.441-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:16:13.652-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:16:13 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:16:13 UPR_STREAMREQ for vb 270 successful
CapiNozzle 2017-04-25T13:16:14.610-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:16:14 UPR_STREAMREQ for vb 75 successful
CapiNozzle 2017-04-25T13:16:14.804-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
StatisticsManager 2017-04-25T13:16:15.749-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998 {"Errors": "[{\"time\":\"2017-04-25T13:16:12.324151353-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 121, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 6536, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 121, "set_filtered": 0, "set_received_from_dcp": 6536, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]", "Status": "Pending"}

2017/04/25 13:16:16 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:16:16 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:16:16 UPR_STREAMREQ for vb 271 successful
CapiNozzle 2017-04-25T13:16:17.022-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:16:17 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:16:17 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:16:18.576-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CheckpointManager 2017-04-25T13:16:18.600-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
CapiNozzle 2017-04-25T13:16:18.602-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CheckpointManager 2017-04-25T13:16:18.600-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:16:18.666-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:16:18 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:16:18 UPR_STREAMREQ for vb 296 successful
DcpNozzle 2017-04-25T13:16:18.725-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:16:18.779-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:18.816-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:16:19.140-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:16:19.151-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:16:19.154-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:16:19.154-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:16:19.407-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:16:19.688-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:16:19.688-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:16:19.688-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:16:19.688-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:16:19.689-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:16:26.690-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:16:26.670-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:16:26.839-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
PipelineManager 2017-04-25T13:16:26.942-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:16:26.944-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:16:26.946-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
GenericSupervisor 2017-04-25T13:16:26.936-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
PipelineRuntimeCtx 2017-04-25T13:16:26.949-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:16:27.019-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:16:26.892-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:16:26.951-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:16:27.114-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:16:27.119-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:27.077150199 -0500 CDT
GenericSupervisor 2017-04-25T13:16:27.077-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:16:27.170-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:16:27.170-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CapiNozzle 2017-04-25T13:16:27.175-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CheckpointManager 2017-04-25T13:16:27.183-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:16:27.184-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:16:27.185-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:16:27.185-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CapiNozzle 2017-04-25T13:16:27.187-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:16:27.194-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:16:27.197-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:16:27.198-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:16:27.198-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:16:27.198-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:16:27.198-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-168372998 has been stopped
GenericPipeline 2017-04-25T13:16:27.215-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-168372998 is stopped
PipelineManager 2017-04-25T13:16:27.215-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:16:27.222-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
GenericPipeline 2017-04-25T13:16:27.224-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
GenericPipeline 2017-04-25T13:16:27.224-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
GenericPipeline 2017-04-25T13:16:27.224-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-168372998
AsyncListener 2017-04-25T13:16:27.225-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:27.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:27.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:27.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:27.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:27.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:27.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:27.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:27.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:16:27.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:27.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:16:27.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:16:27.231-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:16:27.232-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:16:27.235-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:16:27.240-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:16:27.240-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:16:27.242-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:16:27.242-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:16:27.242-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:16:27.242-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:16:27.242-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:16:27.242-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:16:27.242-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:16:27.242-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:27.242229149 -0500 CDT
DcpNozzle 2017-04-25T13:16:27.245-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:16:27.245-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:27.242503041 -0500 CDT
CapiNozzle 2017-04-25T13:16:27.245-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:16:27.245-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:16:27.245-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:16:27.246-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
PipelineManager 2017-04-25T13:16:27.216-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:16:12.324151353-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:16:27.248-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:16:27.293-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:16:27.325-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:16:27.343-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
DcpNozzle 2017-04-25T13:16:27.352-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:16:27.352-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:27.352126305 -0500 CDT
DcpNozzle 2017-04-25T13:16:27.352-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:16:27.352-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:16:27.352-05:00 [INFO] Finish executing 0x65e930
DcpNozzle 2017-04-25T13:16:27.377-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:16:27.352314581 -0500 CDT
DcpNozzle 2017-04-25T13:16:27.377-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:16:27.377-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:27.377-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:16:27.383-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:16:27.388-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:16:27.388-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:27.388-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:16:27.391-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:16:27.391-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:16:27.391-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:16:27.391-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
PipelineManager 2017-04-25T13:16:27.517-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:16:12.324151353-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:16:27.534-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:16:27.539-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:16:27.539-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:16:27.654-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:16:27.657-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:16:27.660-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:16:27.660-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:16:27.660-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:16:27.660-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:16:27.660-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:16:27.662-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:16:27.666-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
GenericSupervisor 2017-04-25T13:16:27.671-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:16:27.671-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
GenericSupervisor 2017-04-25T13:16:27.671-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:27.671-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:27.671-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:27.671-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:16:27.671-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:16:27.675-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
CheckpointManager 2017-04-25T13:16:27.675-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
CapiUtils 2017-04-25T13:16:27.755-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:16:27.756-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:16:27.757-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:16:27.761-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
StatisticsManager 2017-04-25T13:16:27.770-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:16:27.771-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
XDCRFactory 2017-04-25T13:16:27.771-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:16:27.772-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 is constructed, start it
GenericPipeline 2017-04-25T13:16:27.775-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[293 175 208 264 310 260 270 276 290 333 288 307 258 285 174 180 172 326 338 215 330 308 199 232 282 185 203 244 322 256 281 291 197 237 184 287 234 309 337 255 272 296 315 283 324 273 286 299 329 205 216 218 229 261 265 266 243 278 280 295 263 249 334 183 253 321 313 277 221 336 192 240 201 230 319 187 209 269 254 332 335 198 204 242 206 239 300 292 181 259 262 214 304 318 320 301 323 247 294 298 178 220 245 317 202 238 275 188 224 305 196 210 246 267 233 303 213 173 191 252 251 289 302 340 200 228 271 186 193 212 223 325 327 207 312 241 195 219 257 284 182 225 231 189 211 297 306 274 217 222 248 171 331 235 250 268 226 339 194 227 311 179 279 316 328 190 314 176 177 236] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[58 68 95 144 6 128 130 136 143 63 83 146 14 48 72 74 93 114 166 41 53 94 111 139 57 91 141 37 142 13 47 77 112 116 10 17 65 161 33 59 55 99 169 36 38 42 73 159 84 126 29 97 98 86 125 140 167 46 100 121 163 27 89 134 0 15 96 124 31 32 155 49 69 81 2 23 70 44 75 120 168 18 82 145 147 22 35 50 104 123 61 110 21 26 30 52 127 85 115 133 162 34 152 51 157 105 117 54 80 109 149 20 90 19 66 103 151 9 88 101 107 16 92 119 135 8 43 138 153 45 137 25 60 102 7 148 164 12 113 154 1 62 108 5 11 24 158 170 129 156 39 71 4 76 150 160 64 79 40 67 118 132 28 87 122 165 3 78 106 56 131]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[43 138 153 45 137 25 60 102 7 148 164 12 113 154 1 62 108 5 11 24 158 170 129 156 39 71 4 76 150 160 64 79 40 67 118 132 28 87 122 165 3 78 106 56 131 58 68 95 6 128 130 136 143 144 63 83 146 14 48 72 74 93 114 166 41 53 94 111 139 57 91 141 37 142 13 47 77 112 116 10 17 65 161 33 59 55 99 169 36 38 42 73 159 84 126 29 97 98 86 125 140 167 46 100 121 163 27 89 134 0 15 96 124 31 32 155 49 69 81 2 23 70 44 75 120 168 18 82 145 22 35 50 104 123 147 61 110 21 26 30 52 127 85 115 133 162 34 152 51 157 105 117 54 80 109 149 20 90 19 66 103 151 9 88 101 107 16 92 119 135 8] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[200 228 186 193 212 223 271 325 327 207 312 241 195 219 182 225 231 257 284 189 211 297 306 217 222 248 274 171 331 226 235 250 268 194 227 311 339 179 190 279 316 328 176 177 314 236 175 208 293 264 310 260 270 276 290 288 307 333 174 180 258 285 172 326 215 330 338 199 232 282 308 185 203 244 322 197 237 256 281 291 184 287 234 309 337 255 272 296 315 273 283 324 205 216 218 286 299 329 229 243 261 265 266 278 263 280 295 249 334 183 253 321 277 313 221 336 192 240 201 187 209 230 319 254 269 198 204 242 332 335 206 239 300 181 259 262 292 214 304 318 320 301 323 178 220 245 247 294 298 317 202 238 275 188 224 305 196 210 246 233 267 213 303 173 191 252 251 289 302 340]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 log_level:Info replication_type:capi filter_expression: active:true failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:16:27.778-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234...
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:16:27.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:16:27.791-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:16:27.791-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:16:27.792-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:16:27.793-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:16:27.793-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:16:27.793-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:16:27.793-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:16:27.797-05:00 [INFO] Checkpointing starts in 1803.127 sec
PipelineRuntimeCtx 2017-04-25T13:16:27.797-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:16:27.797-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:16:27.798-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:16:27.806-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:16:27.806-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:16:27.807-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:16:27.807-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:16:27.810-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:16:27.848-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:16:27.848-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:16:27.849-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:16:27.855-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:16:27.855-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:16:27.855-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:16:27.861-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:16:27.887-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:16:27.887-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:16:27.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:16:27.900-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:16:27.988-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:16:27.990-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:16:27.990-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:16:27.990-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:16:27.993-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:16:27.994-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:16:27.994-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:16:28.021-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:16:28.022-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:16:28.022-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:16:28.038-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:16:28.038-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:16:28.199-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:16:28.204-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:16:28.204-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:16:28.205-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:16:28.217-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:16:28.217-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:16:28.217-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:16:28.217-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:16:28.217-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:16:28.217-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:16:28.217-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 is started----------
PipelineManager 2017-04-25T13:16:28.217-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:16:28.217-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:16:28.218-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:16:28 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 210 successful
CapiNozzle 2017-04-25T13:16:28.675-05:00 [INFO] Send batch count=28
CapiNozzle 2017-04-25T13:16:28.676-05:00 [INFO] Send batch count=268
2017/04/25 13:16:28 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:16:28 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:16:29 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:16:30 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:16:31 UPR_STREAMREQ for vb 315 successful
CapiNozzle 2017-04-25T13:16:31.847-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:16:32 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:16:32 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:16:32 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:16:32 UPR_STREAMREQ for vb 17 successful
CapiNozzle 2017-04-25T13:16:33.264-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:16:33.392-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:16:34 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:16:34 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:16:34 UPR_STREAMREQ for vb 18 successful
CapiNozzle 2017-04-25T13:16:34.842-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
2017/04/25 13:16:37 UPR_STREAMREQ for vb 316 successful
GenericSupervisor 2017-04-25T13:16:37.928-05:00 [INFO] About to call function
2017/04/25 13:16:38 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:16:38 UPR_STREAMREQ for vb 117 successful
CapiNozzle 2017-04-25T13:16:37.890-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
StatisticsManager 2017-04-25T13:16:38.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:16:38.583-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:16:39.281-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:16:39.981-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:16:39.476-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
GenericSupervisor 2017-04-25T13:16:40.213-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:16:40.691-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:16:41.694-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
2017/04/25 13:16:42 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:16:43 UPR_STREAMREQ for vb 19 successful
CapiNozzle 2017-04-25T13:16:44.303-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:16:44 UPR_STREAMREQ for vb 118 successful
GenericSupervisor 2017-04-25T13:16:46.875-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
2017/04/25 13:16:48 UPR_STREAMREQ for vb 217 successful
CapiNozzle 2017-04-25T13:16:48.172-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
StatisticsManager 2017-04-25T13:16:48.177-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:16:12.324151353-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1280, "dcp_dispatch_time": 2058, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 128, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 5694, "docs_rep_queue": 4715, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 5694, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 128, "set_filtered": 0, "set_received_from_dcp": 5694, "size_rep_queue": 5933522, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 66}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 122, "docs_opt_repd": 0, "docs_rep_queue": 5418, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 122, "size_rep_queue": 8465945, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 215, "max": 69, "mean": 68.4, "min": 68}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 6, "docs_opt_repd": 0, "docs_rep_queue": 5211, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 6, "size_rep_queue": 7069576, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 13, "max": 39, "mean": 39, "min": 39}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 410, "dcp_dispatch_time": {"count": 3592, "max": 828150, "mean": 1418.636, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 7411, "expiry_received_from_dcp": 0, "set_received_from_dcp": 7411}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 365, "dcp_dispatch_time": {"count": 2100, "max": 1709358, "mean": 3152.579, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 7437, "expiry_received_from_dcp": 0, "set_received_from_dcp": 7437}}

StatisticsManager 2017-04-25T13:16:48.182-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 7696 items, sent 268 items
StatisticsManager 2017-04-25T13:16:48.182-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 7437 items, sent 28 items
GenericSupervisor 2017-04-25T13:16:48.052-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:16:48.055-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
2017/04/25 13:16:49 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 318 successful
StatisticsManager 2017-04-25T13:16:49.911-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:7696]
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 8485 items, sent 8484 items. streams inactive: [24 42 157 155 36 129 134 158 167 96 126 127 141 26 53 84 161 147 70 80 83 85 144 150 140 34 48 136 130 170 23 51 67 94 40 54 139 149 21 79 124 160 81 132 50 62 66 135 60 64 133 33 119 125 44 61 71 98 29 43 145 152 30 69 73 63 153 92 169 138 37 137 90 123 27 65 68 55 56 91 93 28 39 46 52 120 148 165 35 49 74 95 41 82 142 143 77 86 168 47 58 31 88 131 163 97 89 122 151 22 75 87 25 166 78 146 57 59 72 76 121 156 164 32 99 128 154 159 38 45 162]
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:7441]
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 7441 items, sent 7440 items. streams inactive: [283 225 189 231 245 328 239 278 263 265 291 187 228 235 246 254 296 326 237 181 199 227 232 333 185 218 226 236 282 240 248 277 322 330 253 255 273 297 261 188 193 238 276 294 325 179 196 230 266 321 176 198 250 286 338 195 222 251 260 268 258 270 318 178 180 220 224 229 320 336 290 184 223 234 244 257 274 275 171 191 242 252 174 183 256 293 334 182 219 259 271 173 241 281 194 284 298 327 177 221 299 323 329 335 172 192 272 186 190 262 269 249 264 295 243 280 288 319 279 292 339 175 247 285 324 331 332 197 233 267 287 289 337 340]
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =788 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =4 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =3 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =788 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
GenericSupervisor 2017-04-25T13:16:49.401-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:16:50.384-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:16:50.399-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:16:50 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 319 successful
CapiNozzle 2017-04-25T13:16:50.769-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:16:50.770-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:16:50.770-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:16:50.772-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]
ReplicationManager 2017-04-25T13:16:50.853-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]
2017/04/25 13:16:50 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 23 successful
StatisticsManager 2017-04-25T13:16:50.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:16:50.909-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:16:12.324151353-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 336, "dcp_dispatch_time": 638, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 128, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 16668, "docs_rep_queue": 15629, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 10974, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 128, "set_filtered": 0, "set_received_from_dcp": 16668, "size_rep_queue": 21470227, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 66}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 122, "docs_opt_repd": 0, "docs_rep_queue": 12884, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 122, "size_rep_queue": 15141095, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 215, "max": 69, "mean": 68.4, "min": 68}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 6, "docs_opt_repd": 0, "docs_rep_queue": 7413, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 6, "size_rep_queue": 10327348, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 13, "max": 39, "mean": 39, "min": 39}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 336, "dcp_dispatch_time": {"count": 9133, "max": 215487, "mean": 231.772, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 13274, "expiry_received_from_dcp": 0, "set_received_from_dcp": 13274}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 7533, "max": 907286, "mean": 1132.459, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 12843, "expiry_received_from_dcp": 0, "set_received_from_dcp": 12843}}

StatisticsManager 2017-04-25T13:16:50.909-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 13273 items, sent 268 items
StatisticsManager 2017-04-25T13:16:50.909-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 13014 items, sent 28 items
StatisticsManager 2017-04-25T13:16:50.909-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:13273]
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 13274 items, sent 13273 items. streams inactive: [92 138 169 37 137 27 65 68 90 123 91 93 28 39 46 52 55 56 165 35 49 74 95 148 41 82 142 143 168 47 58 77 86 31 88 163 97 131 89 151 22 75 87 122 25 166 146 57 59 72 76 78 164 32 99 121 156 154 159 38 45 128 162 24 42 157 36 129 134 155 96 158 167 127 141 26 53 84 126 161 70 80 83 147 85 144 150 34 48 136 140 23 51 67 94 130 170 40 54 139 149 79 124 160 132 50 62 66 81 135 60 64 133 33 125 71 98 29 43 44 61 145 152 30 69 73 63 153]
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:13028]
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 13029 items, sent 13029 items. streams inactive: [274 275 171 191 242 252 174 183 256 334 182 219 259 271 293 173 241 281 194 284 298 327 177 221 299 323 335 172 192 272 329 186 190 262 269 249 264 295 243 280 288 279 292 175 247 285 324 339 331 332 197 233 267 287 289 337 340 283 225 189 231 245 328 239 278 265 291 187 228 235 246 254 263 296 326 181 199 227 232 237 185 226 236 282 333 240 248 277 322 330 253 255 273 297 188 193 238 261 294 325 179 196 230 266 276 176 198 250 286 321 195 222 251 260 268 338 270 178 180 220 224 229 258 320 336 184 223 234 244 257 290]
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =202 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =201 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:16:50.910-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 336, "dcp_dispatch_time": 638, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 128, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 16668, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 128, "set_filtered": 0, "set_received_from_dcp": 16668, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

ToplogyChangeDetector 2017-04-25T13:16:50.912-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:16:50 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:16:50 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 221 successful
PipelineManager 2017-04-25T13:16:51.231-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:16:51.235-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:16:51.249-05:00 [INFO] err_list=[{"time":"2017-04-25T13:16:51.24953141-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries"}]
PipelineManager 2017-04-25T13:16:51.260-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:16:51 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 223 successful
PipelineManager 2017-04-25T13:16:51.407-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:16:51.24953141-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]}
]
2017/04/25 13:16:51 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 227 successful
CapiNozzle 2017-04-25T13:16:51.559-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:16:51 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 30 successful
CapiNozzle 2017-04-25T13:16:51.701-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:16:51 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:16:51 UPR_STREAMREQ for vb 329 successful
CapiNozzle 2017-04-25T13:16:51.885-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:16:52.606-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:16:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:16:57 UPR_STREAMREQ for vb 330 successful
CapiNozzle 2017-04-25T13:16:57.361-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
GenericSupervisor 2017-04-25T13:16:58.062-05:00 [INFO] About to call function
2017/04/25 13:16:59 UPR_STREAMREQ for vb 134 successful
GenericSupervisor 2017-04-25T13:16:59.927-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:17:00 UPR_STREAMREQ for vb 33 successful
GenericSupervisor 2017-04-25T13:16:59.817-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:17:00.678-05:00 [INFO] Executing 0x681fb0 timed out
PipelineManager 2017-04-25T13:17:02.841-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries 
GenericSupervisor 2017-04-25T13:17:03.049-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:17:02.777-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
PipelineManager 2017-04-25T13:17:03.735-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:17:03.749-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:17:03.753-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
GenericSupervisor 2017-04-25T13:17:03.648-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:17:03.879-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:17:03.881-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:17:04.016-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
2017/04/25 13:17:04 UPR_STREAMREQ for vb 331 successful
GenericSupervisor 2017-04-25T13:17:04.043-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:17:04.052-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:17:03.881888771 -0500 CDT
GenericSupervisor 2017-04-25T13:17:03.730-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:18:32.016-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
GenericSupervisor 2017-04-25T13:18:31.874-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:18:32.184-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CAPIService 2017-04-25T13:18:32.192-05:00 [INFO] Received error when making rest call. baseURL=10.202.98.33:9091, path=_pre_replicate, ret_err=Post http://10.202.98.33:9091/_pre_replicate: EOF, statusCode=0, num_retry=5
CheckpointManager 2017-04-25T13:18:32.250-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:18:32.251-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:18:32.252-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:18:32.252-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:18:32.254-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:18:32.254-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-660801234 has been stopped
GenericPipeline 2017-04-25T13:18:32.273-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
GenericPipeline 2017-04-25T13:18:32.274-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
GenericPipeline 2017-04-25T13:18:32.274-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
GenericPipeline 2017-04-25T13:18:32.274-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-660801234
AsyncListener 2017-04-25T13:18:32.275-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:18:32.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:18:32.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:18:32.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:18:32.283-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:18:32.283-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:18:32.284-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:18:32.289-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:18:32.294-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:18:32.295-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:18:32.297-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:18:32.297-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=171
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=172
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=173
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=174
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=175
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=176
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=177
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=178
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=179
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=180
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=181
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=182
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=183
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=184
DcpNozzle 2017-04-25T13:18:32.297-05:00 [INFO] There is no active stream for vb=185
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=186
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=187
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=188
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=189
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=190
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=191
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=192
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=193
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=194
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=195
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=196
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=197
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=198
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] There is no active stream for vb=199
GenericPipeline 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:18:32.298-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:18:32.298-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:18:32.298-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:18:32.298-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:18:32.300-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:18:32.297767156 -0500 CDT
DcpNozzle 2017-04-25T13:18:32.307-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:18:32.307-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:18:32.298250944 -0500 CDT
panic: send on closed channel

goroutine 10011 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc521764240, 0x8edb60, 0xc2b99f4690, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc521759b90, 0x8ed920, 0xc53acbd130, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc521532780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 15 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 15 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 15 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc2080d8370)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc20806e420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc20806e420, 0xc2080de001, 0x5ff, 0x5ff, 0xc2080dc080, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080de001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080d4000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4000, 0x8f07a0, 0xc20806e4a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc20806e480, 0xc2080da5e0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bc00, 0x7fdc3d34ac10, 0xc20806e480, 0x0, 0x0, 0xc2080da5e0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bc00, 0x7fdc3d34ac10, 0xc20806e480, 0xc208084280, 0xc20801f660, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bc00, 0x7fdc3d34ac10, 0xc20806e480)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b5f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b5f90, 0xc2080be480, 0x7fdc3d349398, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 13 [syscall, 15 minutes]:
syscall.Syscall(0x0, 0x0, 0xc20812f000, 0x1000, 0xc20803b1d0, 0xc2080e2320, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc20812f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc20812f000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc20812f000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc20812f000, 0x1000, 0x1000, 0xc2080f5630, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208106c00)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208106c00, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 19 [select, 15 minutes]:
net/http.(*persistConn).readLoop(0xc2081020b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 20 [select, 15 minutes]:
net/http.(*persistConn).writeLoop(0xc2081020b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 14 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc208100380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 9259 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a4b8900, 0x67, 0xc35b15c700, 0xec, 0x100, 0xc4875cb000, 0xb3, 0x100, 0xc40f8a7a40, 0xc40f8a7980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4264 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f258c0, 0x12e, 0xc3c7b9ad00, 0xec, 0x100, 0xc3cd753000, 0xb3, 0x100, 0xc33c082f60, 0xc33c082ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 15 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208100390)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 16 [semacquire]:
sync.(*WaitGroup).Wait(0xc20809c0c0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:191 +0x428
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 33 [select, 15 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 34 [select, 15 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20812dfe0, 0x11, 0xc208159fc0, 0xc2081067e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20812dfe0, 0x11, 0xc208159fc0, 0xc2081067e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20812dfe0, 0x11, 0xc208159fc0, 0xc2081067e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080f4ff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 35 [select, 15 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080d9e80, 0xf, 0xc208158fc0, 0xc2081067e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080d9e80, 0xf, 0xc208158fc0, 0xc2081067e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080d9e80, 0xf, 0xc208158fc0, 0xc2081067e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080f5180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 36 [select]:
net/http.(*persistConn).roundTrip(0xc20802e4d0, 0xc483afad60, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc53b8e6680, 0xc2b99f5a90, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc53b8e6680, 0x7fdc3d348fe8, 0xe94600, 0x49, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc53b8e6680, 0x49, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc53b8e6680, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc53b8e6680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc53b8e6680, 0x7fdc3d34c810, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc53a0d7ce0, 0xc5384339c0, 0x34, 0x7fdc3d34c810, 0xc539ffa000, 0x958760, 0xc53b5e42d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc539fee060, 0xc5384339c0, 0x34, 0x958760, 0xc53b5e42d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Client).GetPool(0xc539fee060, 0x9cae60, 0x7, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:767 +0x339
github.com/couchbase/goxdcr/utils.LocalPool(0xc539b12850, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:89 +0x4a7
github.com/couchbase/goxdcr/utils.LocalBucketUUID(0xc539b12850, 0xe, 0xc2080d9ad8, 0x8, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:370 +0x73
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080e4070, 0xc539b25200, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:464 +0x12d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080e4070, 0xc539b25200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc2080e4a10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 37 [chan send]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208106ba0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:110 +0x737
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 1055 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ab40, 0x67, 0xc287c08700, 0xec, 0x100, 0xc287485800, 0xb3, 0x100, 0xc287cacb40, 0xc287caca80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 38 [runnable]:
net/http.NewRequest(0x9bf4c0, 0x3, 0xc2b4fabec0, 0x21, 0x7fdc3d34caf8, 0xc4d8b84930, 0x40fc48, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:509 +0x1eb
github.com/couchbase/goxdcr/utils.ConstructHttpRequest(0xc24a96a8c0, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:392 +0x396
github.com/couchbase/goxdcr/utils.prepareForRestCall(0xc24a96a8c0, 0x15, 0x9e0430, 0xc, 0xc24a96a800, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:192 +0x173
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc24a96a8c0, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:168 +0x15d
github.com/couchbase/goxdcr/utils.QueryRestApi(0xc24a96a8c0, 0x15, 0x9e0430, 0xc, 0xc208162600, 0x9bf4c0, 0x3, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:134 +0x17a
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostInfo(0xc20801f440, 0xc53b8e8450, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:113 +0x139
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:157 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostName(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:177 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyHost(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:50 +0x4c
github.com/couchbase/goxdcr/replication_manager.GetReplicationInfos(0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:507 +0x304
github.com/couchbase/goxdcr/replication_manager.(*Adminport).doGetAllReplicationInfosRequest(0xc208106ba0, 0xc5393dd6c0, 0xc53be76120, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:361 +0x73
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc208106ba0, 0xc5393dd6c0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:188 +0x7d9
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc208106ba0, 0xc53aa110b0, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc53aa110b0, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208106bb8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 39 [runnable]:
net/http.newBufioWriterSize(0x7fdc3d34c5f0, 0xc5332703c0, 0x1000, 0xc20838ede0)
	/usr/local/go/src/net/http/server.go:488 +0x1d6
net/http.(*Server).newConn(0xc208106de0, 0x7fdc3d349970, 0xc5337ec040, 0xc5332703c0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:444 +0x433
net/http.(*Server).Serve(0xc208106de0, 0x7fdc3d34c590, 0xc208104090, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1746 +0x2f3
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 24 [select, 15 minutes]:
net/http.(*persistConn).readLoop(0xc208102000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 25 [select, 15 minutes]:
net/http.(*persistConn).writeLoop(0xc208102000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 40 [IO wait, 15 minutes]:
net.(*pollDesc).Wait(0xc2080e4450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e4450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e43f0, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc208160180)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208104050, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc208104050, 0xc208102108, 0xc20810a000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080da8c0, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208106360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208106360, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208106360, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080e1200)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080e1200, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080dcc00, 0xc2080df200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080dcc00, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080dcc40, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d4180, 0xc20800bcc0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4180, 0x80f160, 0xc2080dcd00, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 42 [IO wait, 15 minutes]:
net.(*pollDesc).Wait(0xc208010760, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010760, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010700, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc208160330)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380d0, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc2080380d0, 0xc208102058, 0xc2080c3000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20809c240, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806e900)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc20806e900, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc20806e900, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080e1380)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080e1380, 0xc2080df800, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080dce40, 0xc2080df800, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080dce40, 0xc2080df800, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080dce80, 0xc2080df800, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d4480, 0xc20800bf80, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4480, 0x80f160, 0xc2080dcf40, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 538 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x0, 0xc246666700, 0xe9, 0x100, 0xc245e6b200, 0x26, 0x40, 0xc2465eb2c0, 0xc2465eb200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 67 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc208102370)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 66 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc2080e5090, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e5090, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e5030, 0xc2081e0000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc53059f048)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208104120, 0xc2081e0000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc208104120, 0xc2081023c8, 0xc2081e0000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081dffe0, 0xc2081e0000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806ede0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20806ede0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208102370)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 283 [chan send, 15 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x0, 0xc208054700, 0xea, 0x100, 0xc216aa0000, 0xb8, 0x100, 0xc216a83d40, 0xc216a83c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1390 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x1, 0xc247c53700, 0xea, 0x100, 0xc2b3395800, 0xb2, 0x100, 0xc2b39a1740, 0xc2b39a1680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 55 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc2080e52c0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e52c0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e5260, 0xc20812e000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc52ef5e430)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208104140, 0xc20812e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc208104140, 0xc20802e318, 0xc20812e000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20812d6a0, 0xc20812e000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806f560)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20806f560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20802e2c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 56 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc20802e2c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1476 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0xd4, 0xc2ddab8300, 0xec, 0x100, 0xc2da5bd000, 0x1f4, 0x200, 0xc2ddab5740, 0xc2ddab5680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7941 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc498c54240, 0x12c, 0xc2db6f3700, 0xec, 0x100, 0xc4a26eb800, 0x54, 0x80, 0xc4a28783c0, 0xc4a2878300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10171 [runnable]:
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).SetState(0xc52152e9a0, 0x4, 0x13, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:580
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop(0xc52152e9a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:380 +0x6c2
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).stopPipeline(0xeb0fa0, 0xc20802c360, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:341 +0x680
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc5318e0500, 0xc531b7e6c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:583 +0x274
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc5318e0500)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 5209 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3f7166480, 0x64, 0xc40bb2e000, 0xec, 0x100, 0xc40b343800, 0xab, 0x100, 0xc40bb32000, 0xc407875f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1405 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x64, 0xc236f46d00, 0xec, 0x100, 0xc2a9e7c000, 0xc9, 0x100, 0xc2a9fee660, 0xc2a9fee5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4799 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a8b82900, 0x67, 0xc28a082600, 0xec, 0x100, 0xc3997df800, 0x9e, 0x100, 0xc3b482b440, 0xc3b482b380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1805 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x1, 0xc2a098e300, 0xea, 0x100, 0xc29bce6000, 0xc7, 0x100, 0xc2acabc600, 0xc2acabc540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10279 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33e2a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2223 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x2, 0xc348a72a00, 0xea, 0x100, 0xc348afc000, 0xcf, 0x100, 0xc348bb4960, 0xc348bb48a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1931 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x64, 0xc21ccaee00, 0xec, 0x100, 0xc2fd311800, 0xc9, 0x100, 0xc2fd706960, 0xc2fd7068a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4218 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f25680, 0x0, 0xc2ea932b00, 0xea, 0x100, 0xc3453bd800, 0xb8, 0x100, 0xc374dcdc20, 0xc374dcdb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10261 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc53be7d3c0, 0x3b9aca00, 0xc521748ba0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc52152e9a0, 0x7fdc3d352500, 0xc521764240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fdc3d352500, 0xc521764240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 696 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x66, 0xc263042e00, 0xec, 0x100, 0xc262dfb800, 0xc8, 0x100, 0xc2634408a0, 0xc2634407e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 363 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x1, 0xc231bf8e00, 0xea, 0x100, 0xc22caa7000, 0xc7, 0x100, 0xc22f63f920, 0xc22f63f860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 960 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0x130, 0xc278f5cb00, 0xec, 0x100, 0xc279e76000, 0xbf, 0x100, 0xc279f5afc0, 0xc279f5af00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 328 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x2, 0xc211542b00, 0xea, 0x100, 0xc21ca45800, 0xcf, 0x100, 0xc220e6e0c0, 0xc220e6e000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1044 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ab40, 0x1, 0xc27fb5be00, 0xea, 0x100, 0xc281186000, 0xc7, 0x100, 0xc2811d7800, 0xc2811d7740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1878 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x5, 0xc291b51700, 0xea, 0x100, 0xc293404000, 0xbc, 0x100, 0xc2937c18c0, 0xc2937c1800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2761 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a00fc0, 0x64, 0xc30e5d1d00, 0xec, 0x100, 0xc357eef800, 0xc9, 0x100, 0xc282b2eea0, 0xc282b2ede0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2871 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0xcb, 0xc255a30700, 0xec, 0x100, 0xc387435800, 0xc9, 0x100, 0xc3894f68a0, 0xc3894f67e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 358 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x68, 0xc22c670400, 0xec, 0x100, 0xc22b92c800, 0xc2, 0x100, 0xc22c86acc0, 0xc22c86ac00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9985 [runnable]:
encoding/json.(*decodeState).literalInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:937 +0x1ae
encoding/json.(*decodeState).valueInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:836 +0x5c
encoding/json.(*decodeState).objectInterface(0xc53bacc780, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc53bacc780, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc53bacc780, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).objectInterface(0xc53bacc780, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc53bacc780, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc53bacc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc53bacc780, 0xc483afbfa0)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).object(0xc53bacc780, 0x833e80, 0xc483afbfa0, 0xd4)
	/usr/local/go/src/encoding/json/decode.go:497 +0x3c8
encoding/json.(*decodeState).value(0xc53bacc780, 0x812be0, 0xc483afbfa0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).unmarshal(0xc53bacc780, 0x812be0, 0xc483afbfa0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:149 +0x1e7
encoding/json.Unmarshal(0xc52a9d9643, 0x8bd, 0x8bd, 0x812be0, 0xc483afbfa0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:78 +0x145
github.com/couchbase/goxdcr/simple_utils.IsJSON(0xc52a9d9643, 0x8bd, 0x8bd, 0xc2080b73a8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:168 +0x6c
github.com/couchbase/goxdcr/parts.getDocMap(0xc52adf6b40, 0xc53ba78c00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:987 +0x4fe
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc521764240, 0xa900ca, 0xc2080b7b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc521764240, 0xc52b0a00ca, 0xc2080b7b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc521764240, 0xc52a80c700, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc521764240, 0xc52a80c700, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc521764240, 0xc521658c40, 0xc5217643e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 7703 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9440, 0x3, 0xc3447f0f00, 0xea, 0x100, 0xc422d0a000, 0xb5, 0x100, 0xc3a85037a0, 0xc3a85036e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 307 [chan send, 15 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x12d, 0xc21a41e800, 0xec, 0x100, 0xc219943000, 0xcf, 0x100, 0xc21aa8c8a0, 0xc21aa8c7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 304 [chan send, 15 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0xca, 0xc220dd6600, 0xec, 0x100, 0xc2209fa800, 0xbc, 0x100, 0xc221114480, 0xc2211143c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2174 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x0, 0xc30e5d1f00, 0xea, 0x100, 0xc30e956000, 0xb8, 0x100, 0xc30e983860, 0xc30e9837a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6095 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc445770b40, 0x64, 0xc2cb447400, 0xec, 0x100, 0xc3eb99c400, 0x5d, 0x80, 0xc25f298420, 0xc25f298360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1071 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0x12c, 0xc2a098ea00, 0xec, 0x100, 0xc2a0642800, 0xc4, 0x100, 0xc2a0b99b60, 0xc2a0b99aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6555 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715440, 0xc8, 0xc397329c00, 0xec, 0x100, 0xc401048800, 0xbb, 0x100, 0xc3c3b7e900, 0xc3c3b7e840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4050 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f25680, 0x3, 0xc3dca6e600, 0xea, 0x100, 0xc3dd5d4c00, 0x5a, 0x80, 0xc3dd9a5d40, 0xc3dd9a5c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9229 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a7878c0, 0xc9, 0xc3b9d7f900, 0xec, 0x100, 0xc4b6c29000, 0xbd, 0x100, 0xc3c0b1dc20, 0xc3c0b1db60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6539 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715440, 0xc9, 0xc3b8bd9500, 0xec, 0x100, 0xc4356f7800, 0xbd, 0x100, 0xc28657f260, 0xc28657f1a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 95 [runnable]:
net/http.(*persistConn).writeLoop(0xc20802e4d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 9945 [runnable]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc521764000, 0xc521658bd0, 0xc5217641a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 94 [IO wait]:
net.(*pollDesc).Wait(0xc208011090, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011090, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011030, 0xc20c67c000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc53bd95698)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080d60a8, 0xc20c67c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc2080d60a8, 0xc20802e528, 0xc20c67c000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c66bb20, 0xc20c67c000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20c5b79e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20c5b79e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20802e4d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1871 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0xca, 0xc28fdf9800, 0xec, 0x100, 0xc2f1b6f000, 0xbc, 0x100, 0xc2f2638120, 0xc2f2638060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3110 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f680, 0xc8, 0xc211543800, 0xec, 0x100, 0xc395b13000, 0x6f, 0x80, 0xc395e2a0c0, 0xc395e2a000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10260 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc53be7d340, 0x3b9aca00, 0xc521748ba0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc52152e9a0, 0x7fdc3d3528a8, 0xc521532780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fdc3d3528a8, 0xc521532780)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 2399 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44240, 0x1, 0xc3544b0200, 0xea, 0x100, 0xc354584000, 0x3a, 0x40, 0xc354463d40, 0xc354463c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 8089 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc4a8ee8a70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc4a8ee8a70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc4a8ee8a10, 0xc4a9129000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc52f223ae8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc4a6bebe90, 0xc4a9129000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc4a6bebe90, 0xc4a8eabc88, 0xc4a9129000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2d43d9200, 0xc4a9129000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc4a9126840)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc4a9126840, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc4a8eabc30)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2584 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44240, 0x66, 0xc35b15c800, 0xec, 0x100, 0xc3613c3000, 0xc8, 0x100, 0xc2fa9dc300, 0xc2fa9dc240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5573 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc419f598c0, 0xca, 0xc42ca87900, 0xec, 0x100, 0xc4384bb000, 0xbc, 0x100, 0xc438e30c00, 0xc438e30b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3785 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39f08a000, 0x12e, 0xc3c7b9ac00, 0xec, 0x100, 0xc3c7465800, 0xb3, 0x100, 0xc3c7f0cf60, 0xc3c7f0cea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2064 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x64, 0xc3074c6800, 0xec, 0x100, 0xc307877200, 0x3b, 0x40, 0xc3079dcc60, 0xc3079dcba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 378 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x12e, 0xc22d19d800, 0xec, 0x100, 0xc229866800, 0xb3, 0x100, 0xc225696fc0, 0xc225696f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2562 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44240, 0x0, 0xc33aad1000, 0xea, 0x100, 0xc33ea1e800, 0xb8, 0x100, 0xc33f7e5080, 0xc33f7e4fc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 145 [syscall, 15 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 1485 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0xd3, 0xc2dfd1de00, 0xec, 0x100, 0xc2e03b2800, 0xba, 0x100, 0xc2e0729b60, 0xc2e0729aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2186 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x68, 0xc318ce3f00, 0xec, 0x100, 0xc3279f1800, 0xc2, 0x100, 0xc32848bc80, 0xc32848bbc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1876 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0xc9, 0xc291b50700, 0xec, 0x100, 0xc2a971d800, 0xbd, 0x100, 0xc2d71abaa0, 0xc2d71ab9e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 8700 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4c1c8b440, 0x12c, 0xc3ae406e00, 0xec, 0x100, 0xc4ac9dd000, 0xc4, 0x100, 0xc399a04f00, 0xc399a04e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1360 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0xc9, 0xc23cc1e600, 0xec, 0x100, 0xc288c6f000, 0xaa, 0x100, 0xc286cbca20, 0xc286cbc960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2825 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0xc8, 0xc26483af00, 0xec, 0x100, 0xc37cedb000, 0xbb, 0x100, 0xc37d543860, 0xc37d5437a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2860 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a00fc0, 0x69, 0xc246667300, 0xec, 0x100, 0xc38a9a5800, 0xba, 0x100, 0xc38ad66a80, 0xc38ad669c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 389 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x64, 0xc236f46800, 0xec, 0x100, 0xc236992800, 0xc9, 0x100, 0xc2372248a0, 0xc2372247e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1419 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0x12c, 0xc21ccaf200, 0xec, 0x100, 0xc2b4cf3800, 0xc4, 0x100, 0xc2b5838ae0, 0xc2b5838a20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10238 [runnable]:
net/textproto.(*Reader).ReadLine(0xc53be76750, 0xc298a4a1a0, 0x0, 0x7fdc1f721810, 0x0)
	/usr/local/go/src/net/textproto/reader.go:35
net/http.ReadRequest(0xc4cae32ba0, 0xc298a4a1a0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc533744000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc533744000)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2734 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0xcc, 0xc3447f1100, 0xeb, 0x100, 0xc2c1f64000, 0x1f, 0x20, 0xc299ec1140, 0xc299ec1080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2212 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x1, 0xc331b93100, 0xea, 0x100, 0xc33aefc000, 0xc7, 0x100, 0xc33afd70e0, 0xc33afd7020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1268 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x1, 0xc287c08800, 0xe9, 0x100, 0xc24b036f00, 0x1f, 0x20, 0xc27bc2a060, 0xc234a99440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3665 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39f08a000, 0x12d, 0xc3b8bd8400, 0xec, 0x100, 0xc3b7f85000, 0xd0, 0x100, 0xc3b8e94960, 0xc3b8e948a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6791 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc45e460480, 0x12d, 0xc27c0dc100, 0xec, 0x100, 0xc46e5e6000, 0xcc, 0x100, 0xc46e65b6e0, 0xc46e65b620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 722 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316d80, 0xc9, 0xc263042b00, 0xec, 0x100, 0xc24d2f7000, 0xbd, 0x100, 0xc2657d3560, 0xc2657d34a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1459 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x0, 0xc2cb446600, 0xea, 0x100, 0xc2cdffe800, 0xb8, 0x100, 0xc2ceb583c0, 0xc2ceb58300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3200 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f680, 0xc8, 0xc398cece00, 0xec, 0x100, 0xc399fea000, 0x96, 0x100, 0xc399e51aa0, 0xc399e519e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4538 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7440, 0xcf, 0xc33aad1d00, 0xec, 0x100, 0xc33c342800, 0xac, 0x100, 0xc3745998c0, 0xc374599800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 613 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316d80, 0x12e, 0xc22d19df00, 0xec, 0x100, 0xc2329d3800, 0x9d, 0x100, 0xc229f06ea0, 0xc229f06de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 350 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0xcb, 0xc231bf9100, 0xec, 0x100, 0xc23179e800, 0xc9, 0x100, 0xc232342fc0, 0xc232342f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1103 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ab40, 0x65, 0xc211543400, 0xec, 0x100, 0xc2a0641000, 0xc6, 0x100, 0xc29edd3e60, 0xc29edd3da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 336 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x12c, 0xc21ccae300, 0xec, 0x100, 0xc210fd3800, 0xc4, 0x100, 0xc221757d40, 0xc221757c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7611 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9680, 0xd1, 0xc3c7b9b300, 0xec, 0x100, 0xc48060b800, 0xce, 0x100, 0xc4842e00c0, 0xc4842e0000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 691 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316d80, 0xcf, 0xc247c53500, 0xec, 0x100, 0xc25ea82800, 0xac, 0x100, 0xc25f2e8e40, 0xc25f2e8d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 293 [chan send, 15 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0xc9, 0xc211543000, 0xec, 0x100, 0xc21104e000, 0xab, 0x100, 0xc213d0f380, 0xc213d0f2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 315 [chan send, 15 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x66, 0xc21a41fe00, 0xec, 0x100, 0xc2130f6000, 0xc8, 0x100, 0xc2133dfb60, 0xc2133dfaa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2879 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0x131, 0xc278f5d400, 0xec, 0x100, 0xc38b4f5000, 0xbf, 0x100, 0xc38bbc3620, 0xc38bbc3560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1038 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0x12e, 0xc28a082700, 0xec, 0x100, 0xc2896fb000, 0xb3, 0x100, 0xc28a05db00, 0xc28a05da40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 396 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0xc8, 0xc23cc1f000, 0xec, 0x100, 0xc23ce4e000, 0xbb, 0x100, 0xc23ce656e0, 0xc23ce65620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1844 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x2, 0xc2e1536e00, 0xea, 0x100, 0xc28242f000, 0xa8, 0x100, 0xc2c9475da0, 0xc2c9475ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2407 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44480, 0x12c, 0xc3544b0f00, 0xec, 0x100, 0xc354b54000, 0x63, 0x80, 0xc354bc2de0, 0xc354bc2d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10280 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33e360)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2151 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x64, 0xc30c777600, 0xec, 0x100, 0xc31be7f800, 0xac, 0x100, 0xc31c83cea0, 0xc31c83cde0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1447 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x2, 0xc2a098f200, 0xea, 0x100, 0xc2be86a000, 0xcf, 0x100, 0xc2be9d10e0, 0xc2be9d1020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2590 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44480, 0x12e, 0xc3544b1400, 0xec, 0x100, 0xc35e356000, 0xb3, 0x100, 0xc2d354aa20, 0xc2d354a960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5839 [semacquire, 6 minutes]:
sync.(*Mutex).Lock(0xc445ad8868)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc44535f590, 0x109, 0xc35428e000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc44535f590, 0x2, 0xc445dc0710, 0x64, 0x98, 0xc445d13d10, 0xc445651000, 0xc44f057d70, 0xc22522e5c0, 0xc26e6c4d20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 952 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ab40, 0x2, 0xc278f5c500, 0xea, 0x100, 0xc278fb8000, 0xcf, 0x100, 0xc279248360, 0xc2792482a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 704 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316d80, 0x130, 0xc26483b800, 0xec, 0x100, 0xc26bca9000, 0xbc, 0x100, 0xc26c210fc0, 0xc26c210f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1349 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x65, 0xc24932b800, 0xec, 0x100, 0xc293796000, 0xc6, 0x100, 0xc2616d35c0, 0xc2616d3500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1856 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x0, 0xc2db6f3100, 0xea, 0x100, 0xc2d72cb000, 0xb8, 0x100, 0xc26a51e660, 0xc26a51e5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 755 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x64, 0xc236f47300, 0xec, 0x100, 0xc2640b2800, 0xc9, 0x100, 0xc2477e2240, 0xc2477e2180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3205 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f440, 0x64, 0xc39bf00800, 0xec, 0x100, 0xc39bb77800, 0xc9, 0x100, 0xc39c11c300, 0xc39c11c240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1463 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0xd0, 0xc2cef93c00, 0xec, 0x100, 0xc2cf6a8000, 0xb2, 0x100, 0xc2cf6817a0, 0xc2cf6816e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 686 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x3, 0xc21a41e700, 0xea, 0x100, 0xc247cd0800, 0xb5, 0x100, 0xc22b121da0, 0xc22b121ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7289 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464480, 0x12c, 0xc48a304c00, 0xec, 0x100, 0xc48a6e5800, 0xc4, 0x100, 0xc48aa74ba0, 0xc48aa74ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 663 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x65, 0xc247c52800, 0xec, 0x100, 0xc24c674000, 0xc6, 0x100, 0xc24c6b5800, 0xc24c6b5740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 936 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0x130, 0xc220dd7800, 0xec, 0x100, 0xc24def8000, 0x53, 0x80, 0xc24e781e60, 0xc24e781da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10242 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xeb1a20, 0xc53b8e8270, 0xc53b8e82a0, 0xc2080e4690, 0xc52efd8070, 0xc20809c0c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:279 +0x63f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 1042 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0xc9, 0xc27fb5aa00, 0xec, 0x100, 0xc27eec7800, 0xbd, 0x100, 0xc27fcdf380, 0xc27fcdf2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2784 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0x12f, 0xc2dfd1d200, 0xec, 0x100, 0xc3411b5800, 0xce, 0x100, 0xc317808000, 0xc317d91f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 480 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc23c717ce0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 479 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc2419e5720, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2419e5720, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2419e56c0, 0xc241b06000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc530fbae10)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc23d0cb658, 0xc241b06000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc23d0cb658, 0xc23c717d38, 0xc241b06000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc241adaf00, 0xc241b06000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc241add740)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc241add740, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc23c717ce0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2345 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc34f44c8b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc34f44c8b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc34f44c850, 0xc35390c000, 0x1000, 0x1000, 0x0, 0x7fdc3d348df0, 0xc52eda2200)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc34a292eb0, 0xc35390c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fdc3d34ab08, 0xc34a292eb0, 0xc34ad3b7b8, 0xc35390c000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3538fb580, 0xc35390c000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc34f449080)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc34f449080, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc34ad3b760)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1095 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ab40, 0x3, 0xc29f264300, 0xea, 0x100, 0xc29f33e000, 0xb5, 0x100, 0xc29fa14d80, 0xc29fa14cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 629 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x4, 0xc246667900, 0xea, 0x100, 0xc23d251000, 0xa2, 0x100, 0xc2268fcd80, 0xc2268fccc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4747 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a8b82900, 0x67, 0xc2ddab8000, 0xec, 0x100, 0xc37949f800, 0x5d, 0x80, 0xc3ba0584e0, 0xc3ba058420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 718 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316b40, 0x0, 0xc24932ab00, 0xea, 0x100, 0xc266753000, 0xa3, 0x100, 0xc22afad080, 0xc22afacfc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5954 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc445770b40, 0x64, 0xc255a31c00, 0xec, 0x100, 0xc377236400, 0x5d, 0x80, 0xc287503860, 0xc2875037a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1469 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x6, 0xc2db6f3000, 0xea, 0x100, 0xc2da791000, 0xbe, 0x100, 0xc2dc6ecae0, 0xc2dc6eca20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10269 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb06c0, 0x1dcd6500, 0xc521748c60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176e730, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176e730)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2190 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x3, 0xc32ddf7900, 0xea, 0x100, 0xc3316c6000, 0xb5, 0x100, 0xc331770120, 0xc331770060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1409 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0xc8, 0xc22d19c700, 0xec, 0x100, 0xc2aa81e800, 0xbb, 0x100, 0xc2aadebe00, 0xc2aadebd40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1084 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0xc8, 0xc294f75200, 0xec, 0x100, 0xc295458000, 0xbb, 0x100, 0xc29557a180, 0xc29557a0c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1082 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ab40, 0x66, 0xc291b51e00, 0xec, 0x100, 0xc2933a6000, 0xb9, 0x100, 0xc2933f3320, 0xc2933f3260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1854 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0x12d, 0xc2ddab8e00, 0xec, 0x100, 0xc2d805a800, 0xcf, 0x100, 0xc27deb7680, 0xc27deb75c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10236 [runnable]:
io/ioutil.(*nopCloser).Read(0xc53b8ec0d0, 0xc478ee4000, 0x2000, 0x2000, 0xc20802a0d0, 0x93fbc8, 0x7fdc3d34af00)
	<autogenerated>:4
io/ioutil.devNull.ReadFrom(0x0, 0x7fdc3d34cb78, 0xc53b8ec0d0, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/ioutil/ioutil.go:151 +0xb1
io/ioutil.(*devNull).ReadFrom(0xc20802a1e8, 0x7fdc3d34cb78, 0xc53b8ec0d0, 0x7fdc3d34af00, 0x0, 0x0)
	<autogenerated>:9 +0xbb
io.Copy(0x7fdc3d348d18, 0xc20802a1e8, 0x7fdc3d34cb78, 0xc53b8ec0d0, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:358 +0x13d
net/http.(*transferWriter).WriteBody(0xc4d8b84700, 0x7fdc3d34aa58, 0xc539ff8d80, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:219 +0x843
net/http.(*Request).write(0xc53b8e6340, 0x7fdc3d34aa58, 0xc539ff8d80, 0xc53aa11900, 0xc2b4fdef60, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:448 +0xada
net/http.(*persistConn).writeLoop(0xc53a030c60)
	/usr/local/go/src/net/http/transport.go:951 +0x28d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1891 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x66, 0xc27c0dde00, 0xec, 0x100, 0xc2f2e37000, 0xc8, 0x100, 0xc2f35eb800, 0xc2f35eb740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1123 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc257f9ad80, 0xcb, 0xc291b50c00, 0xec, 0x100, 0xc29d0d0000, 0xc9, 0x100, 0xc26c3ea300, 0xc26c3ea240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1483 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8000, 0x67, 0xc2dfd1d500, 0xec, 0x100, 0xc2dfb8e800, 0xb3, 0x100, 0xc2e00204e0, 0xc2e0020420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3713 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39ea2dd40, 0x5, 0xc3ae407700, 0xea, 0x100, 0xc3b8aa4000, 0xaa, 0x100, 0xc3b8b66960, 0xc3b8b668a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 738 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316d80, 0x12c, 0xc24932bd00, 0xec, 0x100, 0xc26499a800, 0xc4, 0x100, 0xc22a17ed20, 0xc22a17ec60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1634 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x2, 0xc2ea932900, 0xea, 0x100, 0xc2eabf2000, 0x43, 0x80, 0xc2eabe8300, 0xc2eabe8240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 638 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23d316d80, 0x12f, 0xc24932aa00, 0xec, 0x100, 0xc248afd800, 0xce, 0x100, 0xc2495372c0, 0xc249537200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2804 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0xcc, 0xc29f264500, 0xec, 0x100, 0xc2b905d000, 0xad, 0x100, 0xc34a2d3d40, 0xc34a2d3c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10263 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0540, 0x1dcd6500, 0xc521748dd0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176f090, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176f090)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1824 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0xc8, 0xc2e2e5ca00, 0xec, 0x100, 0xc29851e800, 0xbb, 0x100, 0xc2d08ea000, 0xc2d0c6bd40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7688 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9440, 0x5, 0xc34ae46b00, 0xea, 0x100, 0xc429d19000, 0xbc, 0x100, 0xc433b56c60, 0xc433b56ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4207 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f258c0, 0x12c, 0xc39bf01e00, 0xec, 0x100, 0xc3d131f800, 0xc4, 0x100, 0xc3dc3c3920, 0xc3dc3c3860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1280 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0xc9, 0xc287c09a00, 0xec, 0x100, 0xc27ab56600, 0x3d, 0x40, 0xc22a2d49c0, 0xc22a2d4900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1650 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0x12c, 0xc2e2e5d700, 0xec, 0x100, 0xc2eacd0400, 0x45, 0x80, 0xc2eaea72c0, 0xc2eaea7200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1451 [chan send, 14 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24edf8240, 0x135, 0xc2bf1bf400, 0xec, 0x100, 0xc2c2914000, 0xb9, 0x100, 0xc2c2964e40, 0xc2c2964d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2533 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44480, 0x12c, 0xc208055300, 0xec, 0x100, 0xc369114800, 0xa3, 0x100, 0xc216bdf7a0, 0xc216bdf6e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2207 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x66, 0xc3447f0a00, 0xec, 0x100, 0xc344755800, 0xc8, 0x100, 0xc344bcf4a0, 0xc344bcf3e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2810 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a00fc0, 0x66, 0xc295ce5700, 0xec, 0x100, 0xc2f55bb000, 0xc8, 0x100, 0xc2f39754a0, 0xc2f39753e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9903 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc52176f400)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 5063 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3f7166480, 0x64, 0xc23d0ee000, 0xec, 0x100, 0xc401e84000, 0x59, 0x80, 0xc40237a780, 0xc40237a6c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10245 [runnable]:
net/http.func·019()
	/usr/local/go/src/net/http/transport.go:519
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:522 +0x335

goroutine 2831 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a00fc0, 0x65, 0xc235929a00, 0xec, 0x100, 0xc380b19000, 0xc6, 0x100, 0xc38151c6c0, 0xc38151c600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1904 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0x12f, 0xc22c670100, 0xec, 0x100, 0xc2fcb27000, 0xce, 0x100, 0xc2fcf3c300, 0xc2fcf3c240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1943 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x64, 0xc21a41fc00, 0xec, 0x100, 0xc2fe1d6000, 0xc9, 0x100, 0xc2fe1e5da0, 0xc2fe1e5ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7484 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464240, 0x1, 0xc3c146bc00, 0xea, 0x100, 0xc40dcb1000, 0xc7, 0x100, 0xc3bcc0db60, 0xc3bcc0daa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4433 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7200, 0x65, 0xc398cede00, 0xec, 0x100, 0xc3bc622800, 0x91, 0x100, 0xc2fe4610e0, 0xc2fe461020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1917 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0x12e, 0xc246667a00, 0xec, 0x100, 0xc2fb564000, 0xb3, 0x100, 0xc2fb58b260, 0xc2fb58b1a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1896 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e12b3d40, 0x4, 0xc23d0ee900, 0xea, 0x100, 0xc2fb87c800, 0xa2, 0x100, 0xc2fbbb5080, 0xc2fbbb4fc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5611 [semacquire, 6 minutes]:
sync.(*WaitGroup).Wait(0xc22522e5c0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc44535f590, 0xc2080dc700, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc2080e2280, 0x7fdc3d3525e0, 0xc44535da20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7fdc3d3525e0, 0xc44535da20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 7835 [semacquire, 4 minutes]:
sync.(*Mutex).Lock(0xc498d86170)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc498bef2c0, 0xb2, 0xc4c1919080, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc498bef2c0, 0x1, 0xc498f21988, 0x64, 0xfc, 0xc2b667fd40, 0xc2db5d4f00, 0xc2b5abf8f0, 0xc320c38be0, 0xc4a1a42ba0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 9219 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a7878c0, 0x12c, 0xc3c7b9b600, 0xec, 0x100, 0xc46ed17000, 0x9c, 0x100, 0xc30bba7ce0, 0xc30bba7c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1797 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e153e000, 0x12c, 0xc294f74400, 0xec, 0x100, 0xc2962ae000, 0x98, 0x100, 0xc296600de0, 0xc296600d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9572 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4d882f8c0, 0x2, 0xc27c0ddd00, 0xea, 0x100, 0xc4ec21b800, 0xaa, 0x100, 0xc4ec5fa6c0, 0xc4ec5fa600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5837 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).UpdateSettings(0xc44565c600, 0xc445d122d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:628 +0x2b7
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).UpdateSettings(0xc44535da20, 0xc445d122d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:633 +0x2b0
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc44535f590, 0x41, 0xc3459b5800, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:392 +0x9b8
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc44535f590, 0x0, 0xc445dc0580, 0x64, 0x160, 0xc445d13d10, 0xc445651000, 0xc44f057d70, 0xc22522e5c0, 0xc26e6c4d20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 8618 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4c1c8b200, 0x0, 0xc2ea933400, 0xea, 0x100, 0xc39d962800, 0xad, 0x100, 0xc2f740a480, 0xc2f740a3c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3176 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f680, 0xc8, 0xc397328800, 0xec, 0x100, 0xc3972f0800, 0x9f, 0x100, 0xc397b4cba0, 0xc397b4cae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2238 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe339b00, 0xc8, 0xc349f87200, 0xec, 0x100, 0xc349837800, 0xbb, 0x100, 0xc34a424ea0, 0xc34a424de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9597 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4d882f8c0, 0x65, 0xc263042c00, 0xec, 0x100, 0xc4f8259800, 0xc6, 0x100, 0xc4f87f94a0, 0xc4f87f93e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2834 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a01200, 0x12c, 0xc291b51600, 0xec, 0x100, 0xc373543800, 0xc4, 0x100, 0xc373eebb60, 0xc373eebaa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2346 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc34ad3b760)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2516 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44240, 0x65, 0xc35bb96200, 0xec, 0x100, 0xc35b96b800, 0xc6, 0x100, 0xc35c1011a0, 0xc35c1010e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7722 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9680, 0xd7, 0xc32ddf7800, 0xec, 0x100, 0xc496324000, 0x1f4, 0x200, 0xc4965456e0, 0xc496545620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4039 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f258c0, 0xc9, 0xc3dca6e300, 0xec, 0x100, 0xc3dd903800, 0x5b, 0x80, 0xc3dd9e93e0, 0xc3dd9e9320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7707 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9680, 0xd4, 0xc331b92900, 0xec, 0x100, 0xc46f46b800, 0xb0, 0x100, 0xc46f475ce0, 0xc46f475c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3090 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f440, 0x0, 0xc220dd7500, 0xe9, 0x100, 0xc3909546c0, 0x8, 0x8, 0xc395959e00, 0xc395959d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9246 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a4b8900, 0x3, 0xc39bf00d00, 0xea, 0x100, 0xc4a8aaf800, 0xb5, 0x100, 0xc2ec7678c0, 0xc2ec767800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6213 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc445770b40, 0x64, 0xc3074c7800, 0xec, 0x100, 0xc331b9b800, 0xaa, 0x100, 0xc31aba8300, 0xc31aba8240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10224 [chan receive, 1 minutes]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208106d80, 0x7fdc3d34c640, 0xc5337448c0, 0xc539e9c5b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208104080, 0x7fdc3d34c640, 0xc5337448c0, 0xc539e9c5b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fdc3d34c640, 0xc5337448c0, 0xc539e9c5b0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208106de0, 0x7fdc3d34c640, 0xc5337448c0, 0xc539e9c5b0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc533271a40)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 10270 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0700, 0x1dcd6500, 0xc521748de0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176f220, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176f220)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2510 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44240, 0x2, 0xc366803400, 0xea, 0x100, 0xc36888e000, 0xcf, 0x100, 0xc3688b7d40, 0xc3688b7c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2789 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a00fc0, 0x0, 0xc2d42a7a00, 0xea, 0x100, 0xc2e0c57800, 0xb6, 0x100, 0xc2d4e68fc0, 0xc2d4e68f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4510 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7440, 0xce, 0xc349c58700, 0xec, 0x100, 0xc3b8838000, 0xab, 0x100, 0xc3761fac60, 0xc3761faba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9888 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc5040f7440, 0xc9, 0xc520a9a100, 0xec, 0x100, 0xc52028d000, 0xbd, 0x100, 0xc520b30780, 0xc520b306c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10129 [runnable]:
bufio.(*Writer).WriteString(0xc44aa80b40, 0x9e1e30, 0xc, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:626
net/http.Header.WriteSubset(0xc53b8e80f0, 0x7fdc3d34c738, 0xc52e8d7f60, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/header.go:156 +0x3e7
net/http.(*chunkWriter).writeHeader(0xc5337445c0, 0xc4c1d35800, 0x3b6, 0x800)
	/usr/local/go/src/net/http/server.go:883 +0xe16
net/http.(*chunkWriter).Write(0xc5337445c0, 0xc4c1d35800, 0x3b6, 0x800, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:256 +0x87
bufio.(*Writer).flush(0xc21bc2cb80, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:530 +0xe0
bufio.(*Writer).Flush(0xc21bc2cb80, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:519 +0x3a
net/http.(*response).finishRequest(0xc5337445a0)
	/usr/local/go/src/net/http/server.go:1018 +0x57
net/http.(*conn).serve(0xc52c209ea0)
	/usr/local/go/src/net/http/server.go:1208 +0xb91
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 8249 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc498c54240, 0x12c, 0xc22d19c600, 0xec, 0x100, 0xc4bb115000, 0xa3, 0x100, 0xc4bb616180, 0xc4bb6160c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5602 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc419f598c0, 0xca, 0xc43f6a5000, 0xec, 0x100, 0xc44344a800, 0xbc, 0x100, 0xc444af2060, 0xc444a6ff80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2248 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fe3398c0, 0x67, 0xc349c58200, 0xec, 0x100, 0xc349410800, 0xb3, 0x100, 0xc349ca4f60, 0xc349ca4ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2552 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44480, 0x130, 0xc32ba48c00, 0xec, 0x100, 0xc345490000, 0xbc, 0x100, 0xc363761f80, 0xc363761ec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4188 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f258c0, 0xc9, 0xc29f264100, 0xec, 0x100, 0xc35d6e9000, 0xa4, 0x100, 0xc3757f0f00, 0xc3757f0e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3770 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39ea2dd40, 0x5, 0xc3c146ab00, 0xea, 0x100, 0xc3c7844000, 0x190, 0x200, 0xc3c8c3cc60, 0xc3c8c3cba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2845 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c6a00fc0, 0x8, 0xc28a082e00, 0xea, 0x100, 0xc378045000, 0xc0, 0x100, 0xc3783d8060, 0xc37823bf80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2506 [chan send, 13 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34ae44480, 0xc8, 0xc35b15d900, 0xec, 0x100, 0xc35e387000, 0xbb, 0x100, 0xc35ea64ea0, 0xc35ea64de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3196 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f440, 0x1, 0xc39857f600, 0xea, 0x100, 0xc397a40800, 0xc7, 0x100, 0xc398a34a80, 0xc398a349c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4841 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a8b82900, 0x65, 0xc287c08d00, 0xec, 0x100, 0xc3e7b43800, 0xc6, 0x100, 0xc3e7d59d40, 0xc3e7d59c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 8473 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4c1c8b200, 0x0, 0xc220dd7600, 0xe9, 0x100, 0xc47e4afb00, 0x11, 0x20, 0xc3db8af320, 0xc3db8aef60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7679 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9680, 0xd3, 0xc349f86c00, 0xec, 0x100, 0xc3b7cdb800, 0xba, 0x100, 0xc41a090ea0, 0xc41a090de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10267 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0640, 0x1dcd6500, 0xc521748e50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176f5e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176f5e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 7432 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464480, 0xc8, 0xc33aad0e00, 0xec, 0x100, 0xc439a0d000, 0xbb, 0x100, 0xc2b50bd680, 0xc2b50bd5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4177 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f25680, 0x3, 0xc278f5d900, 0xea, 0x100, 0xc362e58000, 0x91, 0x100, 0xc28ffa7d40, 0xc28ffa7c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9947 [semacquire]:
sync.(*WaitGroup).Wait(0xc5217641a0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc521764000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc521764000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4512 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7200, 0x67, 0xc349c58e00, 0xec, 0x100, 0xc3b8839800, 0xb3, 0x100, 0xc25f7d0d80, 0xc25f7d0cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4515 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7200, 0x65, 0xc35bb97600, 0xec, 0x100, 0xc3c6496000, 0x9d, 0x100, 0xc2ecea1c80, 0xc2ecea1bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4258 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f25680, 0x1, 0xc3da905c00, 0xea, 0x100, 0xc3ce163800, 0xc7, 0x100, 0xc3455236e0, 0xc345523620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4231 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f25680, 0x64, 0xc331b93800, 0xec, 0x100, 0xc3972ed800, 0xc9, 0x100, 0xc37af783c0, 0xc37af78300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10266 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0600, 0x1dcd6500, 0xc521748e60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176f770, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176f770)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 7836 [semacquire, 4 minutes]:
sync.(*Mutex).Lock(0xc498d86170)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc498bef2c0, 0x105, 0xc4c18db2c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc498bef2c0, 0x2, 0xc498f21a50, 0x64, 0x98, 0xc2b667fd40, 0xc2db5d4f00, 0xc2b5abf8f0, 0xc320c38be0, 0xc4a1a42ba0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3170 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c31f440, 0x0, 0xc396c6b300, 0xea, 0x100, 0xc396254800, 0xb3, 0x100, 0xc396dd3ec0, 0xc396dd3e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7671 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9440, 0x64, 0xc3c9de0b00, 0xec, 0x100, 0xc48252f000, 0xc9, 0x100, 0xc484646ba0, 0xc484646a20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6561 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715200, 0x0, 0xc362a86d00, 0xea, 0x100, 0xc3c7f3e000, 0xb8, 0x100, 0xc37faaed80, 0xc37faaecc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9201 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a4b8900, 0x66, 0xc3c9de1100, 0xec, 0x100, 0xc47da51800, 0xb1, 0x100, 0xc36fbfad20, 0xc36fbfac60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3959 [chan send, 11 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f25680, 0x3, 0xc3da904700, 0xea, 0x100, 0xc3da843800, 0x5a, 0x80, 0xc3da919ce0, 0xc3da919c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6527 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715200, 0x1, 0xc3c146af00, 0xea, 0x100, 0xc440d3d000, 0xc7, 0x100, 0xc37da42b40, 0xc37da42a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5838 [semacquire, 6 minutes]:
sync.(*Mutex).Lock(0xc445ad8868)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc44535f590, 0xa3, 0xc345e0bf80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc44535f590, 0x1, 0xc445dc0648, 0x64, 0xfc, 0xc445d13d10, 0xc445651000, 0xc44f057d70, 0xc22522e5c0, 0xc26e6c4d20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 10258 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc53be7d240, 0x3b9aca00, 0xc521748ba0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc52152e9a0, 0x7fdc3d3528a8, 0xc521532480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fdc3d3528a8, 0xc521532480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 10264 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0580, 0x1dcd6500, 0xc521748cc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176e910, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176e910)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4221 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3d0f258c0, 0xc8, 0xc2e2e5dc00, 0xec, 0x100, 0xc33cf30800, 0xbb, 0x100, 0xc316edef00, 0xc316edee40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9977 [select, 1 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc521532480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 5236 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3f71666c0, 0x12c, 0xc4132b6b00, 0xec, 0x100, 0xc412eeb800, 0xc4, 0x100, 0xc4135254a0, 0xc4135253e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4519 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7440, 0xcb, 0xc3544b0d00, 0xec, 0x100, 0xc3c53c1000, 0xad, 0x100, 0xc280bc1c80, 0xc280bc1bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4584 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7440, 0xcc, 0xc30865cb00, 0xec, 0x100, 0xc374c46000, 0xb7, 0x100, 0xc375236f00, 0xc375236e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4581 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7440, 0xcc, 0xc30865c000, 0xec, 0x100, 0xc3bc6bc000, 0xb7, 0x100, 0xc3bc4bf020, 0xc3bc4bef60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4557 [chan send, 9 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3077f7200, 0x68, 0xc32ddf7100, 0xec, 0x100, 0xc33c341000, 0xc2, 0x100, 0xc2f4f20f00, 0xc2f4f20e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10277 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc521764240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc521764240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e2c7a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 8090 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc4a8eabc30)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 9940 [semacquire, 1 minutes]:
sync.(*WaitGroup).Wait(0xc52a70e5e0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc5218965a0, 0xc2080dc700, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc2080e2280, 0x7fdc3d3525e0, 0xc52152e9a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7fdc3d3525e0, 0xc52152e9a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 6439 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715440, 0x12c, 0xc43b080300, 0xec, 0x100, 0xc42ceee000, 0xc2, 0x100, 0xc4359b3ce0, 0xc4359b3c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10005 [sleep]:
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc5241c6007, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8690, 0xd, 0xc2080d86a0, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:313 +0x5a0
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080d9cf0, 0x9ed550, 0xe, 0xc53b0e3620, 0xeb0d80, 0x5, 0x66504c, 0x30000c5337ec280, 0xc52198d4c0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080d9cf0, 0xc52165a930, 0xc53aef49c0, 0xc47e9c9e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc5218965a0, 0x3, 0xc5219dad58, 0x29, 0x34, 0xc5241c61e0, 0xc52145db00, 0xc525f43170, 0xc52a70e5e0, 0xc52a70c4e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 7121 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464240, 0x0, 0xc247c53400, 0xea, 0x100, 0xc47ed4b800, 0x81, 0x100, 0xc47f1f22a0, 0xc47f1f21e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9987 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc525f42d50, 0xc52a667c20, 0xc52a667c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 7476 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464480, 0x12d, 0xc39ebd7d00, 0xec, 0x100, 0xc3e38ac000, 0xcf, 0x100, 0xc3be2ffbc0, 0xc3be2ffb00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7467 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464240, 0x64, 0xc39ebd7500, 0xec, 0x100, 0xc4672e9800, 0xc9, 0x100, 0xc3b7da6420, 0xc3b7da6360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5917 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc445770d80, 0x12c, 0xc294f74200, 0xec, 0x100, 0xc2f0ddd800, 0x8c, 0x100, 0xc29c762a20, 0xc29c762960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9813 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc5040f7200, 0x2, 0xc50eb1c200, 0xea, 0x100, 0xc50e59f000, 0xa7, 0x100, 0xc50eb25a40, 0xc50eb25980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10003 [select]:
net/http.(*Transport).getConn(0xc20802c090, 0xc53b8e6750, 0x0, 0xc2b4f9a2a0, 0x4, 0xc2b4f9a2a7, 0x11, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:525 +0x608
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc53b8e6750, 0xc2b4f9aa50, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:228 +0x4d4
net/http.send(0xc53b8e6750, 0x7fdc3d348fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc53b8e6750, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc53b8e6750, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc53b8e6750, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc53b8e6750, 0x0, 0x818220, 0xc52eca41b8, 0xeb0d80, 0xc2080d9ce0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc5241c6007, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8690, 0xd, 0xc2080d86a0, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080d9cf0, 0x9ed550, 0xe, 0xc4cae32180, 0xeb0d80, 0x5, 0x66504c, 0x30000c5337ec020, 0xc5219803c0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080d9cf0, 0xc52165a930, 0xc53be7cb60, 0xc208164e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc5218965a0, 0x1, 0xc5219dabc8, 0x64, 0xfc, 0xc5241c61e0, 0xc52145db00, 0xc525f43170, 0xc52a70e5e0, 0xc52a70c4e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 6576 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715440, 0xca, 0xc3544b1d00, 0xec, 0x100, 0xc3dd391000, 0xbc, 0x100, 0xc4063ae660, 0xc4063ae5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10265 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb05c0, 0x1dcd6500, 0xc521748cd0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176eaf0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176eaf0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 9284 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a7878c0, 0x12f, 0xc349c58f00, 0xec, 0x100, 0xc4cf17b000, 0xce, 0x100, 0xc4cffa4a80, 0xc4cffa49c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7658 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38c2b9440, 0x0, 0xc43b081d00, 0xea, 0x100, 0xc480900000, 0xb7, 0x100, 0xc488b69260, 0xc488b691a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 8551 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4c1c8b440, 0xc8, 0xc208055800, 0xeb, 0x100, 0xc441d85200, 0x24, 0x40, 0xc293b26f60, 0xc293b26ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9986 [semacquire]:
sync.(*WaitGroup).Wait(0xc5217643e0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc521764240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc521764240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 6549 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715200, 0x66, 0xc398cec300, 0xec, 0x100, 0xc42ebc8000, 0xc8, 0x100, 0xc3f4baa1e0, 0xc3f4baa120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 8642 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4c1c8b440, 0xc8, 0xc30e5d0c00, 0xec, 0x100, 0xc38b20f800, 0xa0, 0x100, 0xc31ae60f60, 0xc31ae60ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 7418 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc474464240, 0x0, 0xc3074c7000, 0xea, 0x100, 0xc402348000, 0xa5, 0x100, 0xc32b689e60, 0xc32b689da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9890 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc5040f7200, 0x2, 0xc51a8e1700, 0xea, 0x100, 0xc52028a000, 0xb5, 0x100, 0xc5209be360, 0xc5209be2a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 6493 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715440, 0x12d, 0xc3db4a8a00, 0xec, 0x100, 0xc3b8e39000, 0xcf, 0x100, 0xc3e4a7d080, 0xc3e4a7cfc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10220 [chan receive, 1 minutes]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208106d80, 0x7fdc3d34c640, 0xc533744820, 0xc5393dd6c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208104080, 0x7fdc3d34c640, 0xc533744820, 0xc5393dd6c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fdc3d34c640, 0xc533744820, 0xc5393dd6c0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208106de0, 0x7fdc3d34c640, 0xc533744820, 0xc5393dd6c0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc5332719a0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 10275 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc521764000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc521764000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e2a9ec0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 9904 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc52176e370)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 6597 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a9715200, 0x65, 0xc349c59200, 0xec, 0x100, 0xc45d670000, 0xc6, 0x100, 0xc45d6fc960, 0xc45d6fc8a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9303 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a4b8900, 0x65, 0xc32ba49a00, 0xec, 0x100, 0xc4d7802000, 0xc6, 0x100, 0xc4d7c860c0, 0xc4d7c86000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9978 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc521532480, 0xc525c70150)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 7834 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).UpdateSettings(0xc498994d80, 0xc2b679e570, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:628 +0x2b7
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).UpdateSettings(0xc498ac8790, 0xc2b679e570, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:633 +0x2b0
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc498bef2c0, 0x47, 0xc4c16fdce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:392 +0x9b8
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc498bef2c0, 0x0, 0xc498f218c0, 0x64, 0x160, 0xc2b667fd40, 0xc2db5d4f00, 0xc2b5abf8f0, 0xc320c38be0, 0xc4a1a42ba0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 10278 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33e0c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10262 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0500, 0x1dcd6500, 0xc521748dc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176eeb0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176eeb0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 10268 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0680, 0x1dcd6500, 0xc521748c50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176e550, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176e550)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 10002 [runnable]:
github.com/couchbase/go-couchbase.ParseURL(0xc24a96a820, 0x18, 0xc5241c6007, 0x11, 0xc24a96a820)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/util.go:42
github.com/couchbase/goxdcr/utils.ConstructHttpRequest(0xc5241c6007, 0x11, 0x9ed550, 0xe, 0xc53b2b9300, 0xc2080d8690, 0xd, 0xc2080d86a0, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:376 +0xdb
github.com/couchbase/goxdcr/utils.prepareForRestCall(0xc5241c6007, 0x11, 0x9ed550, 0xe, 0x9da000, 0xc2080d8690, 0xd, 0xc2080d86a0, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:192 +0x173
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc5241c6007, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8690, 0xd, 0xc2080d86a0, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:297 +0x1b5
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080d9cf0, 0x9ed550, 0xe, 0xc47e2b8420, 0xeb0d80, 0x5, 0x66504c, 0x30000c52eca40c0, 0xc5218f8a80, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080d9cf0, 0xc52165a930, 0xc24a96a720, 0xc2080bbe00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc5218965a0, 0x0, 0xc5219dab00, 0x64, 0x160, 0xc5241c61e0, 0xc52145db00, 0xc525f43170, 0xc52a70e5e0, 0xc52a70c4e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 8678 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4c1c8b200, 0x64, 0xc366802d00, 0xec, 0x100, 0xc49220a000, 0xc9, 0x100, 0xc3e6f48300, 0xc3e6f48240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10241 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208106d80, 0x7fdc3d34c640, 0xc5332705a0, 0xc292d7be10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208104080, 0x7fdc3d34c640, 0xc5332705a0, 0xc292d7be10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fdc3d34c640, 0xc5332705a0, 0xc292d7be10)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208106de0, 0x7fdc3d34c640, 0xc5332705a0, 0xc292d7be10)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc533270320)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 7791 [semacquire, 4 minutes]:
sync.(*WaitGroup).Wait(0xc320c38be0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc498bef2c0, 0xc2080dc700, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc2080e2280, 0x7fdc3d3525e0, 0xc498ac8790, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7fdc3d3525e0, 0xc498ac8790, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 9787 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc5040f7440, 0x12d, 0xc23d0ef200, 0xeb, 0x100, 0xc50d866300, 0x19, 0x20, 0xc50d844f00, 0xc50d844e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9268 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc40a7878c0, 0xca, 0xc39857f400, 0xec, 0x100, 0xc4aaa91000, 0xbc, 0x100, 0xc4adc4d8c0, 0xc4adc4d800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10239 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208106d80, 0x7fdc3d34c640, 0xc533270460, 0xc292d7ab60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208104080, 0x7fdc3d34c640, 0xc533270460, 0xc292d7ab60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fdc3d34c640, 0xc533270460, 0xc292d7ab60)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208106de0, 0x7fdc3d34c640, 0xc533270460, 0xc292d7ab60)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc5332701e0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 10240 [runnable]:
net/http.newBufioWriterSize(0x7fdc3d34c618, 0xc533270520, 0x800, 0x0)
	/usr/local/go/src/net/http/server.go:488 +0x1d6
net/http.(*conn).readRequest(0xc533270280, 0xc533270500, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:604 +0x4eb
net/http.(*conn).serve(0xc533270280)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 9948 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc525eb1320, 0xc525ece5a0, 0xc525ece600)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 9594 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4d882fb00, 0x12d, 0xc26483be00, 0xec, 0x100, 0xc4f6194800, 0xcf, 0x100, 0xc4f7087440, 0xc4f7087380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10009 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc521532780, 0xc52a0e8e40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 9532 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc4d882fb00, 0xc9, 0xc287c08a00, 0xec, 0x100, 0xc4e8c87000, 0xae, 0x100, 0xc4e95984e0, 0xc4e9598420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10271 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0740, 0x1dcd6500, 0xc521748ce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176ecd0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176ecd0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 10192 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc531c2cc00, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x221
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc531c2cc00, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc52fdf96c0, 0x2, 0x9cac80, 0x3, 0xc530f69aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 9980 [runnable]:
github.com/couchbase/goxdcr/parts.(*Router).route(0xc52138dc20, 0x8ed920, 0xc53af876b0, 0xc2080c4a80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:134 +0x145
github.com/couchbase/goxdcr/parts.*Router.(github.com/couchbase/goxdcr/parts.route)·fm(0x8ed920, 0xc53af876b0, 0xc203d57009, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:69 +0x4d
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc521759a10, 0x8ed920, 0xc53af876b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:57 +0xe9
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc521532480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 9946 [runnable]:
encoding/json.(*decodeState).objectInterface(0xc53bacc4b0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x25a
encoding/json.(*decodeState).valueInterface(0xc53bacc4b0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc53bacc4b0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc53bacc4b0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc53bacc4b0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc53bacc4b0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).objectInterface(0xc53bacc4b0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc53bacc4b0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc53bacc4b0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc53bacc4b0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc53bacc4b0, 0xc483871950)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).object(0xc53bacc4b0, 0x833e80, 0xc483871950, 0xd4)
	/usr/local/go/src/encoding/json/decode.go:497 +0x3c8
encoding/json.(*decodeState).value(0xc53bacc4b0, 0x812be0, 0xc483871950, 0x16)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).unmarshal(0xc53bacc4b0, 0x812be0, 0xc483871950, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:149 +0x1e7
encoding/json.Unmarshal(0xc5213aae43, 0xc10, 0xc10, 0x812be0, 0xc483871950, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:78 +0x145
github.com/couchbase/goxdcr/simple_utils.IsJSON(0xc5213aae43, 0xc10, 0xc10, 0xc52a71f3a8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:168 +0x6c
github.com/couchbase/goxdcr/parts.getDocMap(0xc50e326f60, 0xc53bac4d00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:987 +0x4fe
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc521764000, 0xa90003, 0xc52a71fb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc521764000, 0xc50dd80003, 0xc52a71fb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc521764000, 0xc525c64cb0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc521764000, 0xc525c64cb0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc521764000, 0xc521658af0, 0xc5217641a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 10004 [chan send]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).UpdateSettings(0xc521532780, 0xc5219aa990, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:628 +0x2b7
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).UpdateSettings(0xc52152e9a0, 0xc5219aa990, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:633 +0x2b0
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc5218965a0, 0xe7, 0xc2b4fab680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:392 +0x9b8
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc5218965a0, 0x2, 0xc5219dac90, 0x64, 0x98, 0xc5241c61e0, 0xc52145db00, 0xc525f43170, 0xc52a70e5e0, 0xc52a70c4e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 10259 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc53be7d2c0, 0x3b9aca00, 0xc521748ba0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc52152e9a0, 0x7fdc3d352500, 0xc521764000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fdc3d352500, 0xc521764000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 10276 [runnable]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc525fac7e0, 0xb5d300c9, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc521532780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc521532780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e2c6c60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10208 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc521764240, 0xca, 0xc5304c0e00, 0xec, 0x100, 0xc531c60800, 0xaa, 0x100, 0xc5337c2b40, 0xc5337c2a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 10010 [runnable]:
io.ReadFull(0x7fdc3d34ab08, 0xc52140f020, 0xc4a9dba000, 0xf45, 0xf45, 0xf45, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc2e12bae78, 0x7fdc3d34ab08, 0xc52140f020, 0xc52a72e2a0, 0x18, 0x18, 0xc4c0ab6b1f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc525fac7e0, 0xc52a70d020)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 10008 [select, 1 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc521532780)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 9844 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc5040f7440, 0x12d, 0xc51014a500, 0xec, 0x100, 0xc50f92c800, 0xc0, 0x100, 0xc510516180, 0xc5105160c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 9984 [runnable]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc521764240, 0xc521658d20, 0xc5217643e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 10111 [chan receive, 1 minutes]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc52fdf96c0, 0x9cac80, 0x3, 0xc2080d9ad8)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc521493a40, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc521493a40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc52dc94de0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 9979 [runnable]:
io.ReadFull(0x7fdc3d34ab08, 0xc52140efd8, 0xc4ca638000, 0x3a1, 0x3a1, 0x3a1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc2e12b8e78, 0x7fdc3d34ab08, 0xc52140efd8, 0xc525e87fe0, 0x18, 0x18, 0xc4caad201f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc525e92630, 0xc525ded800)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 10273 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb07c0, 0x1dcd6500, 0xc521748c40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176e370, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176e370)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 10274 [runnable]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc525e92630, 0xb5d30001, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc521532480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc521532480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e2a9e00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10272 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc483bb0780, 0x1dcd6500, 0xc521748e40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc52176f400, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fdc3d352a08, 0xc52176f400)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 10281 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33e5a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10282 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33ec00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10283 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33ef00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10284 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33f1a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10285 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33f740)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10286 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e33faa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10287 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e3500c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10288 [chan receive]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e3502a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 10289 [chan receive]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc47e350600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182
[goport] 2017/04/25 13:18:33 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:18:34.238-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:18:34.238-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:18:34.238-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:18:34.274-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:18:34.278-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:18:34.278-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:18:34.278-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:18:34.278-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:18:34.278-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:18:34.279-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:18:34.279-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:18:34.279-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:18:34.279-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:18:34.279-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:18:34.279-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:18:34.279-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:18:34.279-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:18:34.279-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:18:34.279-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:18:34.279-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:18:34.279-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:18:34.280-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:18:34.280-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:18:34.280-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:18:34.285-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:18:34.285-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:18:34.285-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
RemoteClusterService 2017-04-25T13:18:34.285-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
PipelineManager 2017-04-25T13:18:34.394-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:18:49.673-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:18:49.685-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:18:49.789-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:18:49.790-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:18:49.790-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:18:49.895-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:18:49.896-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:18:49.897-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:18:49.897-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:18:49.897-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:18:49.897-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:18:49.898-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:18:49.899-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:18:49.903-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
GenericSupervisor 2017-04-25T13:18:49.905-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:18:49.905-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
GenericSupervisor 2017-04-25T13:18:49.905-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:18:49.905-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:18:49.905-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:18:49.905-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:18:49.905-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:18:49.905-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
CheckpointManager 2017-04-25T13:18:49.905-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
CapiUtils 2017-04-25T13:18:49.937-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:18:49.938-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:18:49.938-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:18:49.938-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
StatisticsManager 2017-04-25T13:18:49.939-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:18:49.939-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
XDCRFactory 2017-04-25T13:18:49.939-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:18:49.939-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531 is constructed, start it
GenericPipeline 2017-04-25T13:18:49.941-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[271 312 327 263 249 252 335 174 208 228 237 274 332 179 199 272 330 248 258 275 309 216 173 192 282 298 244 288 306 311 328 331 203 317 182 177 183 186 229 255 190 218 290 314 305 269 187 222 325 333 194 196 334 313 200 201 184 221 227 256 277 284 320 338 236 226 185 234 293 307 322 215 241 254 304 323 326 195 242 259 212 276 260 278 286 318 209 336 243 250 253 266 291 296 188 206 265 299 301 337 213 268 238 316 230 175 181 178 319 180 197 176 294 300 302 214 281 224 264 308 210 297 251 273 283 285 295 204 207 205 232 339 211 220 324 329 245 202 267 310 321 171 217 225 235 270 303 198 246 261 280 193 189 247 289 231 292 172 239 257 191 219 240 262 340 223 233 287 315 279] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[130 135 119 145 59 156 1 8 166 113 144 115 147 160 163 27 73 23 83 128 93 67 85 139 24 74 137 170 31 40 43 107 12 30 84 103 3 14 63 15 69 64 90 88 157 77 101 131 141 148 164 10 46 37 49 25 134 165 89 96 142 57 118 100 4 154 39 112 125 102 123 97 146 11 114 65 94 78 104 18 33 52 91 95 2 61 50 143 66 92 136 26 48 53 126 161 21 60 129 47 108 105 138 159 81 98 87 110 116 121 16 75 36 70 22 151 168 7 162 109 158 19 38 106 32 51 34 42 62 72 6 13 68 120 122 71 82 133 150 155 149 79 111 29 86 45 152 58 169 56 153 17 44 127 99 140 9 80 124 20 55 0 167 41 54 117 5 35 132 28 76]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[213 268 238 316 230 175 181 178 319 180 197 176 294 300 302 214 281 224 264 308 297 210 283 285 295 251 273 204 207 205 232 339 324 329 211 220 245 202 321 267 310 171 217 303 225 235 270 280 193 198 246 261 247 289 189 172 231 292 257 239 240 262 191 219 340 223 233 287 315 279 271 312 327 263 249 252 335 237 274 332 174 208 228 179 199 272 330 309 216 248 258 275 298 173 192 282 311 328 331 203 244 288 306 182 317 183 186 229 177 255 218 290 314 190 269 305 333 187 222 325 194 196 334 200 201 313 256 277 284 184 221 227 320 338 226 236 185 234 215 293 307 322 241 254 326 304 323 195 212 242 259 276 278 286 318 260 209 336 253 266 291 243 250 296 299 301 337 188 206 265] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[26 48 66 92 136 53 126 21 60 161 47 108 129 81 98 105 138 159 121 16 75 87 110 116 36 70 22 151 7 162 168 19 38 109 158 32 51 106 72 6 13 34 42 62 68 120 71 82 122 133 150 155 79 111 149 29 86 45 152 58 169 17 44 56 153 127 9 80 99 140 20 55 124 0 167 5 35 41 54 117 28 76 132 130 135 119 145 59 156 1 8 113 144 166 163 27 73 115 147 160 23 83 128 93 67 24 74 85 139 31 40 137 170 12 30 43 107 3 14 84 103 63 15 69 64 90 88 157 141 148 164 10 46 77 101 131 37 49 25 89 96 134 165 57 118 142 100 4 154 39 112 102 123 125 97 146 11 65 94 114 18 33 78 104 52 91 2 61 95 50 143]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[replication_type:capi filter_expression: active:true failure_restart_interval:10 optimistic_replication_threshold:256 stats_interval:1000 checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:18:49.941-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531...
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:18:49.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:18:49.959-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:18:49.959-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:18:49.959-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:18:49.959-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:18:49.959-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:18:49.959-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:18:49.959-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:18:49.959-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:18:49.959-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:18:49.959-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:18:49.959-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:18:49.959-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:18:49.959-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:18:49.959-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:18:49.961-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:18:49.961-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:18:49.961-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:18:49.961-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:18:49.961-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
StatisticsManager 2017-04-25T13:18:50.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
TCPConnPoolMgr 2017-04-25T13:18:50.130-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:18:50.130-05:00 [INFO] connectionSize=5
ToplogyChangeDetector 2017-04-25T13:18:50.131-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:18:50.132-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:18:50.132-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:18:50.132-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:18:50.132-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:18:50.132-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:18:50.133-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:18:50.133-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CheckpointManager 2017-04-25T13:18:50.134-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:18:50.135-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:18:50.135-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:18:50.193-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:18:50.194-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:18:50.194-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:18:50.194-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:18:50.194-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:18:50.194-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:18:50.282-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:18:50.282-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:18:50.282-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:18:50.293-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:18:50.301-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:18:50.301-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:18:50.456-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:18:50.457-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:18:50.457-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:18:50.457-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:18:50.457-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:18:50.457-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:18:50.457-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531 is started----------
PipelineManager 2017-04-25T13:18:50.457-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:18:50.457-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:18:50.457-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:18:50.536-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:18:50.537-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:18:50.537-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:18:50.537-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:18:50 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:18:50.724-05:00 [INFO] Send batch count=2
2017/04/25 13:18:50 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:18:50.732-05:00 [INFO] Send batch count=385
2017/04/25 13:18:50 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 10 successful
CapiNozzle 2017-04-25T13:18:50.853-05:00 [INFO] Send batch count=187
2017/04/25 13:18:50 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:18:50 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 51 successful
StatisticsManager 2017-04-25T13:18:51.028-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:18:51 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 57 successful
CapiNozzle 2017-04-25T13:18:51.336-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:18:51 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 63 successful
CapiNozzle 2017-04-25T13:18:51.353-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:18:51 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 168 successful
CapiNozzle 2017-04-25T13:18:51.546-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:51.556-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:18:51 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 186 successful
CheckpointManager 2017-04-25T13:18:51.689-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
CheckpointManager 2017-04-25T13:18:51.689-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:18:51.696-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:18:51 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:18:51 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:18:51.864-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:18:51.880-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:18:51.892-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
StatisticsManager 2017-04-25T13:18:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:18:52.359-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:52.370-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:18:52.838-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:18:52.849-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:18:53.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:18:53.123-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:53.160-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:18:53.704-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:18:53.704-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:18:53.704-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:18:53.704-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:18:53.704-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:18:53.707-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:18:53.707-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:18:53.710-05:00 [INFO] err_list=[{"time":"2017-04-25T13:18:53.710497571-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:18:53.710-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:18:53.839-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:53.839-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:18:53.840-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:18:53.840-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:18:53.840-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:18:53.872-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:53.872-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:53.883-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:18:53.886-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:53.886-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:53.905-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
StatisticsManager 2017-04-25T13:18:53.962-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:18:53.963-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531 {"Errors": "[{\"time\":\"2017-04-25T13:18:53.710497571-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 272, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 44124, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 272, "set_filtered": 0, "set_received_from_dcp": 44124, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:18:54.137-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:54.163-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:18:54.409-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:18:54.427-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:18:54.455-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:54.466-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:18:54.811-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:18:54.811-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:18:54.811-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:18:54.811-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:18:54.811-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:18:54.883-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:54.883-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:54.897-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:18:55.252-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:18:55.284-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:18:55.623-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:18:55.637-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:18:56.317-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:18:56.317-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:18:56.317-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:18:56.317-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:18:56.317-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:18:56.404-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:56.404-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:56.421-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:18:56.701-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:18:56.714-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:18:57.035-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:18:57.052-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:18:57.450-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:18:57.450-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:18:57.450-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:18:57.450-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:18:57.450-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:18:57.553-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:57.553-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:57.569-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:18:57.891-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:18:57.914-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:18:58.144-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:18:58.155-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:18:58.461-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:18:58.461-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:18:58.461-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:18:58.461-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:18:58.461-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:18:58.529-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:58.529-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:58.541-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:18:58.850-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:18:58.865-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:18:59.196-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:18:59.219-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:18:59.713-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:18:59.713-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:18:59.713-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:18:59.713-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:18:59.714-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:18:59.829-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:18:59.829-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:18:59.852-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
GenericSupervisor 2017-04-25T13:18:59.966-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:18:59.966-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:18:59.966-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:19:00.294-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
ToplogyChangeDetector 2017-04-25T13:19:00.294-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:19:00.307-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:19:01.267-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:01.287-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:19:01.562-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:19:01.562-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:19:01.562-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:19:01.562-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:01.562-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:01.674-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:01.674-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:01.695-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:19:02.178-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:02.195-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:19:02.360-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:19:02.371-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:19:02.517-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:02.517-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:19:02.517-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:19:02.517-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:02.517-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:02.642-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:02.642-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:02.660-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:19:02.801-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:19:02.811-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:19:03.063-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:03.074-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:19:03.589-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:19:03.589-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:19:03.589-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:19:03.589-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:03.589-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:03.688-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:03.688-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:03.699-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
PipelineManager 2017-04-25T13:19:03.711-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:19:03.711-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:19:03.711-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:03.711-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
PipelineRuntimeCtx 2017-04-25T13:19:03.711-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:19:03.711-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:03.711-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:19:03.711-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:03.711609941 -0500 CDT
GenericSupervisor 2017-04-25T13:19:03.711-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:03.711-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:19:03.711-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:03.711-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:19:03.711-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:03.711-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:19:03.711-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:19:03.712-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:19:03.712-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:19:03.712-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-898010531 has been stopped
GenericPipeline 2017-04-25T13:19:03.712-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-898010531 is stopped
PipelineManager 2017-04-25T13:19:03.712-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:19:03.712-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:18:53.710497571-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:19:03.712-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:03.712-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
GenericPipeline 2017-04-25T13:19:03.712-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
GenericPipeline 2017-04-25T13:19:03.712-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
GenericPipeline 2017-04-25T13:19:03.712-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-898010531
AsyncListener 2017-04-25T13:19:03.712-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:03.712-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:03.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:19:03.713-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:03.713-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:19:03.713-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:19:03.713-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:03.713-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:19:03.713-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:19:03.713-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:03.713-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:19:03.713-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:19:03.713-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:03.713-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:19:03.713-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:03.714-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:03.714-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:03.713632458 -0500 CDT
DcpNozzle 2017-04-25T13:19:03.714-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:19:03.714-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:03.713940796 -0500 CDT
DcpNozzle 2017-04-25T13:19:03.714-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:19:03.714-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:03.715-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:19:03.715-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:03.715-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:03.715-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:03.716-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:03.716-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:03.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:19:03.716-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:03.716-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:19:03.718-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:03.718-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:03.719-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:03.718984292 -0500 CDT
DcpNozzle 2017-04-25T13:19:03.719-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:03.718999968 -0500 CDT
DcpNozzle 2017-04-25T13:19:03.719-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:19:03.719-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:19:03.719-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:03.719-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:19:03.719-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:03.719-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:19:03 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:19:03 Stream Closed for vb 171 StreamEnd simulated
CapiNozzle 2017-04-25T13:19:03.780-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:19:03.780-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:19:03.780-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:03.781-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:19:03.841-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:03.842-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:19:03.842-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:19:03.893-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:03.894-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:19:03.894-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:03.894-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:19:03.894-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:03.894-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:19:03.894-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:19:03.894-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:19:03.895-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
GenericSupervisor 2017-04-25T13:19:03.897-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:19:03.897-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
GenericSupervisor 2017-04-25T13:19:03.897-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:03.897-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:03.897-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:03.897-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:03.897-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:19:03.897-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
CheckpointManager 2017-04-25T13:19:03.897-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
CapiUtils 2017-04-25T13:19:03.929-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:03.929-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:03.930-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:19:03.931-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
StatisticsManager 2017-04-25T13:19:03.931-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:19:03.931-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
XDCRFactory 2017-04-25T13:19:03.931-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:19:03.932-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585 is constructed, start it
GenericPipeline 2017-04-25T13:19:03.933-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[150 56 118 70 169 134 19 22 128 131 31 45 120 89 101 88 97 104 3 44 9 109 117 115 136 14 113 139 69 42 63 160 16 142 121 164 25 37 162 10 35 168 43 119 50 87 154 140 148 29 96 125 132 46 62 78 85 146 6 8 129 34 114 52 80 82 84 138 170 73 77 155 163 67 151 5 48 127 167 66 75 58 124 51 53 81 144 24 27 122 153 161 40 92 17 47 28 39 86 149 54 33 49 111 130 23 105 7 20 93 145 59 1 123 64 147 55 116 99 100 112 57 94 165 2 158 133 143 159 72 26 60 102 166 36 90 141 38 110 108 65 74 11 12 76 83 106 157 4 30 68 71 152 0 61 18 91 21 135 103 126 137 41 95 107 13 15 32 156 79 98] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[340 197 270 272 285 295 318 334 176 198 200 221 303 335 305 220 171 191 206 237 246 256 325 230 308 331 257 224 314 216 175 177 306 312 222 264 328 231 259 273 180 184 205 243 229 300 307 187 297 241 330 178 213 332 336 209 215 219 299 250 266 242 268 304 189 248 279 282 315 227 236 254 269 309 316 260 217 265 185 239 252 290 320 194 207 181 244 173 292 322 202 232 319 188 225 310 317 211 277 192 234 337 210 240 263 182 193 258 313 247 249 280 324 190 286 278 284 288 329 338 199 204 245 218 281 291 212 262 183 196 302 179 276 298 226 267 275 327 321 214 301 172 208 253 274 294 195 203 233 235 251 326 333 283 293 311 228 201 271 296 238 323 339 223 186 261 287 174 255 289]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[92 122 153 161 40 47 17 39 28 149 86 49 54 33 105 111 130 23 20 7 145 93 59 123 1 147 64 116 55 94 99 100 112 57 158 165 2 143 133 159 60 72 26 166 102 90 36 110 141 38 74 108 65 12 11 30 76 83 106 157 4 61 68 71 152 0 91 18 21 126 135 103 95 137 41 107 13 32 15 98 156 79 118 150 56 169 70 22 134 19 45 128 131 31 101 120 89 44 88 97 104 3 9 117 109 113 115 136 14 139 63 69 42 142 160 16 37 121 164 25 35 162 10 119 168 43 87 50 154 96 140 148 29 132 125 8 46 62 78 85 146 6 114 129 34 52 77 80 82 84 138 170 73 151 155 163 67 48 5 75 127 167 66 53 58 124 51 27 81 144 24] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[173 292 322 202 232 319 188 225 310 317 211 277 192 234 337 182 210 240 263 193 258 313 247 249 280 324 190 286 278 284 288 329 338 199 204 245 218 281 291 212 262 183 196 302 179 276 298 226 267 275 327 301 321 214 172 208 253 294 274 195 203 233 235 251 326 333 228 283 293 311 201 271 296 238 323 223 339 186 261 287 174 255 289 340 197 270 272 285 295 318 334 176 198 200 221 303 335 305 171 220 191 206 237 246 256 325 230 308 331 257 224 314 216 175 177 306 312 222 264 328 231 259 273 180 184 205 243 187 229 300 307 297 213 241 330 178 332 336 209 215 219 299 242 250 266 268 304 189 248 279 282 315 227 236 254 269 309 316 260 217 239 265 185 207 252 290 320 194 181 244]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 optimistic_replication_threshold:256 stats_interval:1000 replication_type:capi filter_expression: checkpoint_interval:1800 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info active:true worker_batch_size:500 failure_restart_interval:10]
CheckpointManager 2017-04-25T13:19:03.933-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585...
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:03.941-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:19:03.941-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:03.941-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:19:03.941-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:19:03.941-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:19:03.941-05:00 [INFO] Checkpointing starts in 1801.023 sec
PipelineRuntimeCtx 2017-04-25T13:19:03.941-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:19:03.941-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:19:03.941-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:19:03.941-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:19:03.945-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:19:03.945-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:19:03.945-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:19:03.945-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:19:03.949-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:19:04.213-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:19:04.215-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:04.215-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:04.216-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:19:04.246-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:19:04.257-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:19:04.329-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:19:04.376-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:19:04.376-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:19:04.376-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:19:04.392-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:19:04.392-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:19:04.465-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:19:04.465-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:19:04.465-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:19:04.465-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
PipelineManager 2017-04-25T13:19:04.469-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:18:53.710497571-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:19:04.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:19:04.469-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:04.469-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:19:04.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:19:04.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:19:04.469-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:19:04.487-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:19:04.487-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:19:04.487-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:19:04.499-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:19:04.499-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:19:04.649-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:19:04.649-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:04.649-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:19:04.649-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:19:04.649-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:19:04.649-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:19:04.649-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585 is started----------
PipelineManager 2017-04-25T13:19:04.650-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:19:04.650-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:19:04.650-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:19:04 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:19:04.710-05:00 [INFO] Send batch count=76
CapiNozzle 2017-04-25T13:19:04.712-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:19:04.735-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:04.735-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:04.751-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:19:04 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 9 successful
CapiNozzle 2017-04-25T13:19:04.972-05:00 [INFO] Send batch count=500
2017/04/25 13:19:04 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:19:04 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 20 successful
StatisticsManager 2017-04-25T13:19:05.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:19:05 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 118 successful
CapiNozzle 2017-04-25T13:19:05.102-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:05.138-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:19:05 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 82 successful
CheckpointManager 2017-04-25T13:19:05.939-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
CheckpointManager 2017-04-25T13:19:05.939-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:19:05 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:19:05 UPR_STREAMREQ for vb 94 successful
StatisticsManager 2017-04-25T13:19:05.991-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:19:06.012-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:06.034-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
DcpNozzle 2017-04-25T13:19:06.054-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:19:06.072-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:19:06.567-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:06.582-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:19:06.772-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:06.772-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:19:06.772-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:19:06.772-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:19:06.772-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:19:06.782-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:19:06.782-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:19:06.784-05:00 [INFO] err_list=[{"time":"2017-04-25T13:19:06.784978065-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:19:06.785-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:19:06.874-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:06.874-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:06.903-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
StatisticsManager 2017-04-25T13:19:06.954-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:19:06.954-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585 {"Errors": "[{\"time\":\"2017-04-25T13:19:06.784978065-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 336, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 4, "docs_received_from_dcp": 53718, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 336, "set_filtered": 0, "set_received_from_dcp": 53718, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:19:07.539-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:07.554-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:19:07.861-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:07.886-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:19:08.623-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:08.642-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:19:09.212-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:09.263-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:19:10.053-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:10.053-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:19:10.053-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:19:10.053-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:10.053-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:10.146-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:10.146-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:10.168-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:19:10.528-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:10.543-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:19:10.543-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:19:10.543-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:10.543-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:10.641-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:10.641-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:10.656-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:19:11.521-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:11.537-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:19:12.026-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:12.042-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:19:12.947-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:12.958-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:19:13.589-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:13.600-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
GenericSupervisor 2017-04-25T13:19:13.942-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:19:13.942-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:19:13.942-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:19:14.353-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:14.353-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:19:14.353-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:19:14.353-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:14.353-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:19:14.360-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:19:14.443-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:14.443-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:14.454-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:19:14.863-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:14.863-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:19:14.863-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:19:14.863-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:14.863-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:14.926-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:14.926-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:14.937-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:19:15.901-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:19:15.912-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:19:16.363-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:16.374-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
PipelineManager 2017-04-25T13:19:16.785-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:19:16.785-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:19:16.785-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:16.785-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
PipelineRuntimeCtx 2017-04-25T13:19:16.785-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:19:16.785-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:16.785-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:19:16.785-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:16.785617065 -0500 CDT
GenericSupervisor 2017-04-25T13:19:16.785-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:16.785-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:19:16.785-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:16.785-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:19:16.785-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:16.785-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:19:16.785-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:19:16.785-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:19:16.785-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:19:16.785-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-894701585 has been stopped
GenericPipeline 2017-04-25T13:19:16.786-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894701585 is stopped
PipelineManager 2017-04-25T13:19:16.786-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:19:16.786-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:06.784978065-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:19:16.786-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:16.786-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
GenericPipeline 2017-04-25T13:19:16.786-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
GenericPipeline 2017-04-25T13:19:16.786-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
GenericPipeline 2017-04-25T13:19:16.786-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-894701585
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:16.786-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:16.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:16.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:16.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:16.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:19:16.787-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:16.787-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:19:16.787-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:19:16.787-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:16.787-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:19:16.787-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:19:16.787-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:16.787-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:19:16.787-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:19:16.787-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:16.787-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:19:16.787-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:19:16.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:16.788-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:16.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:19:16.788-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:16.787663535 -0500 CDT
DcpNozzle 2017-04-25T13:19:16.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:19:16.788-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:16.787976029 -0500 CDT
CapiNozzle 2017-04-25T13:19:16.788-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:19:16.788-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:16.789-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:19:16.789-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:16.789-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:19:16.789-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:19:16.789-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.789-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:16.789-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.789-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:16.789-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:16.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:16.790-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:19:16.790-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:16.790-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:19:16.795-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:16.795-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:16.795-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:16.795637032 -0500 CDT
DcpNozzle 2017-04-25T13:19:16.795-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:16.795655225 -0500 CDT
DcpNozzle 2017-04-25T13:19:16.795-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:19:16.795-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:19:16.795-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:16.795-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:19:16.795-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:16.795-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:19:16 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:19:16 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:19:16.889-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:16.889-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:19:16.889-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:19:16.979-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:16.979-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:19:16.979-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:16.979-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:19:16.979-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:16.979-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:19:16.980-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:19:16.980-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:19:16.980-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
GenericSupervisor 2017-04-25T13:19:16.981-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:19:16.981-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
GenericSupervisor 2017-04-25T13:19:16.981-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:16.981-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:16.981-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:16.981-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:16.981-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:19:16.981-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
CheckpointManager 2017-04-25T13:19:16.981-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
CapiUtils 2017-04-25T13:19:17.043-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:17.043-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:17.043-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:19:17.044-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
StatisticsManager 2017-04-25T13:19:17.044-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:19:17.044-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
XDCRFactory 2017-04-25T13:19:17.044-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:19:17.045-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271 is constructed, start it
GenericPipeline 2017-04-25T13:19:17.045-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[142 2 69 85 96 121 101 107 122 129 135 168 19 133 24 114 160 5 28 77 111 145 148 98 22 41 54 70 14 110 26 31 56 131 11 20 53 59 92 93 127 35 84 113 153 33 43 158 170 21 80 134 140 150 57 66 91 109 37 47 61 106 152 63 23 45 60 6 126 9 115 105 88 100 156 10 75 32 144 76 112 119 128 30 108 125 44 71 99 0 3 83 147 40 146 7 79 12 58 151 29 55 120 74 118 165 166 1 49 155 8 51 137 167 18 102 116 117 52 73 138 154 13 38 104 163 164 72 89 130 162 46 34 103 139 157 27 97 15 62 86 16 36 90 124 48 81 169 67 78 95 143 123 17 68 82 87 25 64 94 149 4 50 159 141 42 65 132 136 39 161] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[209 233 234 269 275 284 314 274 336 307 331 192 230 249 287 318 191 268 289 327 212 225 239 216 219 270 303 211 316 339 259 309 312 181 330 293 263 272 290 306 173 188 201 323 329 203 285 286 305 308 260 282 296 178 187 228 185 224 248 292 337 262 176 238 317 291 189 232 264 237 280 184 222 295 304 333 207 208 300 313 325 198 206 276 196 199 301 302 229 179 226 217 227 320 338 202 205 277 193 218 235 334 182 190 213 255 279 271 335 278 297 250 265 281 246 324 326 204 210 247 242 256 180 267 221 223 299 321 197 186 220 194 253 322 311 175 315 231 245 266 195 200 215 332 172 183 254 174 258 298 319 340 251 283 310 243 244 252 257 273 288 328 177 214 171 240 261 294 236 241]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[34 103 139 157 27 97 15 62 86 16 36 90 124 48 81 67 78 95 143 169 17 68 82 87 123 25 64 94 149 4 50 159 42 65 132 136 141 39 161 2 69 85 96 142 121 101 107 122 129 135 168 19 133 24 114 160 5 28 77 111 145 148 22 41 54 70 98 14 110 26 31 56 131 11 20 53 59 92 93 127 35 84 113 153 33 43 158 21 80 134 140 170 57 66 91 109 150 37 47 61 106 152 63 23 45 60 6 126 9 115 105 88 100 156 10 75 32 76 112 119 128 144 30 108 125 44 71 99 0 3 83 147 40 146 7 79 12 58 151 29 55 120 74 118 165 166 1 49 155 8 51 18 102 116 117 137 167 52 73 138 154 13 38 104 72 89 130 162 163 164 46] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[175 311 315 195 200 215 231 245 266 172 183 332 254 319 174 258 298 340 251 283 310 243 244 273 252 257 288 328 171 177 214 240 261 236 241 294 234 209 233 269 275 284 314 274 336 307 331 192 230 249 287 191 268 289 318 212 327 225 239 216 303 219 270 211 316 339 259 309 312 181 330 293 306 173 188 201 263 272 290 323 329 305 203 285 286 308 178 260 282 296 187 228 248 185 224 292 337 176 238 262 317 189 232 264 291 184 237 280 333 207 208 222 295 304 198 206 300 313 325 276 196 199 301 302 179 226 229 217 227 320 338 193 202 205 277 182 218 235 334 279 190 213 255 271 335 250 265 278 297 246 281 204 324 326 210 247 242 256 267 180 321 221 223 299 197 220 186 194 253 322]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi active:true checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 filter_expression: doc_batch_size_kb:2048 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:19:17.045-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271...
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:17.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:17.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:17.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:17.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:17.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:19:17.054-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:17.054-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:19:17.054-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:19:17.054-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:19:17.054-05:00 [INFO] Checkpointing starts in 1801.233 sec
PipelineRuntimeCtx 2017-04-25T13:19:17.054-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:19:17.054-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:19:17.054-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:19:17.054-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:19:17.058-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:19:17.058-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:19:17.058-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:19:17.058-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:19:17.058-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:19:17.073-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:19:17.073-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:19:17.073-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:19:17.089-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:19:17.089-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:19:17.103-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:17.103-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:17.103-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:19:17.114-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:19:17.116-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:19:17.117-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:19:17.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:19:17.128-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:17.128-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:19:17.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:19:17.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:19:17.129-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:19:17.146-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:19:17.146-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:19:17.146-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:19:17.165-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:19:17.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:19:17.259-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:19:17.276-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:19:17.276-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:17.276-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:19:17.276-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:19:17.276-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:19:17.277-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:19:17.277-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271 is started----------
PipelineManager 2017-04-25T13:19:17.277-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:19:17.277-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:19:17.277-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:19:17.467-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:19:17.467-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:19:17.467-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:19:17.467-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:19:17 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:19:17.595-05:00 [INFO] Send batch count=31
2017/04/25 13:19:17 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:19:17.643-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:17.643-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:17.646-05:00 [INFO] Send batch count=121
2017/04/25 13:19:17 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:19:17.673-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:19:17.679-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:17.679-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:19:17 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:19:17.713-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:19:17 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:19:17 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 322 successful
StatisticsManager 2017-04-25T13:19:18.099-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:19:18 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 45 successful
CapiNozzle 2017-04-25T13:19:18.253-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:19:18 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 50 successful
CapiNozzle 2017-04-25T13:19:18.265-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:19:18 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 57 successful
CapiNozzle 2017-04-25T13:19:18.355-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:18.383-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:19:18 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 298 successful
CheckpointManager 2017-04-25T13:19:18.807-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
CheckpointManager 2017-04-25T13:19:18.807-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:19:18 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:19:18 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:19:18.934-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:19:18.980-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:19:19.081-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:19.123-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:19:19.140-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
PipelineManager 2017-04-25T13:19:19.368-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:19:19.437-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:19.456-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:19:19.650-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:19.650-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:19:19.650-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:19:19.650-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:19:19.650-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:19:19.653-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:19:19.653-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:19:19.655-05:00 [INFO] err_list=[{"time":"2017-04-25T13:19:19.655757244-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:19:19.655-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:19:19.739-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:19.739-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:19.770-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:19:20.015-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:20.015-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:19:20.015-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:19:20.015-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:20.015-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:19:20.060-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:19:20.060-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271 {"Errors": "[{\"time\":\"2017-04-25T13:19:19.655757244-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 64, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 62977, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 64, "set_filtered": 0, "set_received_from_dcp": 62977, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:19:20.120-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:20.120-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:20.137-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:19:20.370-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:19:20.386-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:19:20.622-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:20.642-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:19:21.739-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:19:21.743-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:21.765-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:19:21.779-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:19:22.347-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:22.347-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:19:22.347-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:19:22.347-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:22.347-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:22.430-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:22.430-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:22.457-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:19:22.525-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:19:22.525-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:19:22.525-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:19:22.525-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:22.525-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:22.593-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:22.594-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:22.610-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:19:23.591-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:19:23.608-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:19:23.870-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:23.888-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:19:24.715-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:19:24.728-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:19:25.528-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:25.544-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:19:26.096-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:19:26.096-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:19:26.096-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:19:26.097-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:26.097-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:26.179-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:26.179-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:26.190-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:19:26.320-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:26.320-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:19:26.320-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:19:26.320-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:26.320-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:26.414-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:26.414-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:26.425-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:19:26.820-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:26.831-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
GenericSupervisor 2017-04-25T13:19:27.054-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:19:27.054-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:19:27.054-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:19:27.306-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:19:27.461-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:27.472-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:19:27.895-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:27.906-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:19:28.230-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:28.240-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:19:28.606-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:28.606-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:19:28.606-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:19:28.606-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:28.606-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:28.684-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:28.684-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:28.695-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:19:28.950-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:28.950-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:19:28.950-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:19:28.950-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:28.951-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:29.002-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:29.003-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:29.013-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:19:29.387-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:19:29.397-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
PipelineManager 2017-04-25T13:19:29.656-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:19:29.656-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:19:29.656-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:29.656-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
PipelineRuntimeCtx 2017-04-25T13:19:29.656-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:19:29.656-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:29.656-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:19:29.656-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:29.656413126 -0500 CDT
GenericSupervisor 2017-04-25T13:19:29.656-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:29.656-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:19:29.656-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:29.656-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:19:29.656-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:29.656-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:19:29.656-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:19:29.656-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:19:29.656-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:19:29.656-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-980022271 has been stopped
GenericPipeline 2017-04-25T13:19:29.656-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-980022271 is stopped
PipelineManager 2017-04-25T13:19:29.657-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
PipelineManager 2017-04-25T13:19:29.657-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:19.655757244-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:19:29.657-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-980022271
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:19:29.657-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:19:29.657-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:19:29.657-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:19:29.657-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:29.657-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:29.657-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:19:29.657-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:29.657-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:19:29.658-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:19:29.658-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:29.657583995 -0500 CDT
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:19:29.658-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:19:29.658-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:19:29.658-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:29.657958896 -0500 CDT
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.658-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:19:29.658-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:19:29.659-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:29.659-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:19:29.659-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:19:29.659-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:19:29.659-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:19:29.659-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:19:29.659-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:29.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:19:29.659-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:29.660-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:19:29.660-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:29.660-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:19:29.660-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:29.660-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:29.660308594 -0500 CDT
DcpNozzle 2017-04-25T13:19:29.660-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:19:29.660-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:29.660-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:19:29.661-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:29.661-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:29.661729948 -0500 CDT
DcpNozzle 2017-04-25T13:19:29.661-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:19:29.661-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:29.661-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:19:29 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:19:29 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:19:29 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:19:29.762-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:29.762-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:19:29.763-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:19:29.866-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:29.867-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:19:29.867-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:29.867-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:19:29.867-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:29.867-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:19:29.867-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:19:29.868-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:19:29.868-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
GenericSupervisor 2017-04-25T13:19:29.870-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:19:29.870-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
GenericSupervisor 2017-04-25T13:19:29.870-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:29.870-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:29.870-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:29.870-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:29.870-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:19:29.870-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
CheckpointManager 2017-04-25T13:19:29.870-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
CapiUtils 2017-04-25T13:19:29.947-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:29.947-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:29.947-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:19:29.948-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
StatisticsManager 2017-04-25T13:19:29.949-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:19:29.949-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
XDCRFactory 2017-04-25T13:19:29.949-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:19:29.949-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301 is constructed, start it
GenericPipeline 2017-04-25T13:19:29.950-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[57 130 165 105 150 40 138 140 157 49 66 129 52 101 123 152 1 6 95 133 145 32 44 132 147 148 85 117 8 162 17 65 89 67 97 99 18 43 68 135 169 2 61 146 163 170 106 110 4 16 47 72 80 50 84 139 15 35 104 23 112 59 120 70 76 77 136 29 46 107 111 124 113 9 22 31 60 109 100 156 160 41 64 125 11 25 96 7 128 34 5 24 58 71 154 103 121 142 153 20 38 39 88 94 114 92 3 81 30 14 48 87 161 0 26 74 79 141 21 51 144 27 155 115 75 98 126 131 73 53 122 149 137 82 108 118 119 134 10 36 167 143 158 164 62 63 69 91 102 19 56 86 127 78 159 28 83 116 151 54 90 166 13 33 93 12 37 42 55 45 168] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[205 330 247 248 333 256 278 292 228 305 222 232 291 176 198 283 312 314 265 273 321 326 173 225 233 296 313 323 329 223 279 187 206 202 252 282 229 324 258 304 319 182 195 251 320 208 224 236 239 243 235 293 197 210 298 311 215 217 259 267 216 301 295 192 300 327 218 260 263 174 177 274 191 213 234 171 189 337 306 307 211 237 240 286 339 340 196 207 318 254 172 316 209 231 308 194 190 261 179 201 262 284 310 185 193 253 325 269 178 200 245 183 175 270 184 199 226 250 328 249 285 331 255 276 309 180 244 317 334 257 315 219 220 288 289 335 212 281 246 280 294 297 188 336 241 264 181 242 299 332 214 238 186 303 268 302 338 230 277 287 227 290 322 204 275 271 272 203 221 266]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[3 81 30 14 48 87 161 0 26 74 79 141 21 51 144 27 155 115 75 98 126 131 73 53 122 149 137 82 108 118 119 134 10 36 167 143 158 164 62 63 69 91 102 19 56 86 127 78 159 28 83 116 151 54 90 166 13 33 93 12 37 42 55 45 168 57 130 165 105 150 40 138 140 157 49 66 129 52 101 123 152 1 6 95 133 145 32 44 132 147 148 85 117 8 162 17 65 89 67 97 99 18 43 68 135 169 2 61 146 163 170 106 110 4 16 47 72 80 50 84 139 15 35 104 23 112 59 120 70 76 77 136 29 46 107 111 124 113 9 22 31 60 109 100 156 160 41 64 125 11 25 96 7 128 34 5 24 58 71 154 103 121 142 153 20 38 39 88 94 114 92] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[269 178 200 245 183 175 270 184 199 226 250 328 249 285 331 255 276 309 180 244 317 334 257 315 219 220 288 289 335 212 281 246 280 294 297 188 336 241 264 181 242 299 332 214 238 186 303 268 302 338 230 277 287 227 290 322 204 275 271 272 203 221 266 205 330 247 248 333 256 278 292 228 305 222 232 291 176 198 283 312 314 265 273 321 326 173 225 233 296 313 323 329 223 279 187 206 202 252 282 229 324 258 304 319 182 195 251 320 208 224 236 239 243 235 293 197 210 298 311 215 217 259 267 216 301 295 192 300 327 218 260 263 174 177 274 191 213 234 171 189 337 306 307 211 237 240 286 339 340 196 207 318 254 172 316 209 231 308 194 190 261 179 201 262 284 310 185 193 253 325]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[optimistic_replication_threshold:256 worker_batch_size:500 doc_batch_size_kb:2048 active:true checkpoint_interval:1800 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: stats_interval:1000]
CheckpointManager 2017-04-25T13:19:29.950-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301...
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:19:29.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:19:29.961-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:29.961-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:19:29.961-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:19:29.961-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:19:29.961-05:00 [INFO] Checkpointing starts in 1803.259 sec
PipelineRuntimeCtx 2017-04-25T13:19:29.961-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:19:29.961-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:19:29.962-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:19:29.962-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:19:29.963-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:19:29.963-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:19:29.963-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:19:29.963-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:19:29.963-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:19:29.988-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:19:29.988-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:19:29.988-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:19:30.006-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:19:30.006-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:19:30.034-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:19:30.035-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:19:30.063-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:30.063-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:30.063-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:19:30.081-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:19:30.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:19:30.083-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:30.083-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:19:30.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:19:30.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:19:30.083-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:19:30.111-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:19:30.111-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:19:30.111-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:19:30.141-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:19:30.141-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:19:30.250-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:19:30.273-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:19:30.273-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:30.273-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:19:30.273-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:19:30.273-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:19:30.273-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:19:30.273-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301 is started----------
PipelineManager 2017-04-25T13:19:30.273-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:19:30.273-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:19:30.274-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:19:30.464-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:19:30.464-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:19:30.464-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:19:30.464-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:19:30 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:19:30.555-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:19:30.558-05:00 [INFO] Send batch count=45
CapiNozzle 2017-04-25T13:19:30.569-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:30.569-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:30.590-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:19:30 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:19:30.631-05:00 [INFO] Send batch count=95
CapiNozzle 2017-04-25T13:19:30.657-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:30.657-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:30.667-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:19:30 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:19:30 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 143 successful
StatisticsManager 2017-04-25T13:19:31.049-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:19:31 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 160 successful
CapiNozzle 2017-04-25T13:19:31.273-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:19:31 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 272 successful
CapiNozzle 2017-04-25T13:19:31.289-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:19:31 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 179 successful
CheckpointManager 2017-04-25T13:19:31.573-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
CheckpointManager 2017-04-25T13:19:31.573-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:19:31.579-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:19:31 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:19:31 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:19:31.622-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
DcpNozzle 2017-04-25T13:19:31.675-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:19:31.685-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:19:32.040-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:19:32.121-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:19:32.141-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:19:32.672-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:32.696-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:19:32.753-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:19:32.753-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:19:32.753-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:19:32.753-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
ReplicationManager 2017-04-25T13:19:32.754-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
PipelineManager 2017-04-25T13:19:32.762-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:19:32.762-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:19:32.762-05:00 [INFO] err_list=[{"time":"2017-04-25T13:19:32.762733919-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]
PipelineManager 2017-04-25T13:19:32.762-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:19:32.854-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:32.854-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:32.867-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
StatisticsManager 2017-04-25T13:19:32.963-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:19:32.964-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301 {"Errors": "[{\"time\":\"2017-04-25T13:19:32.762733919-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 57, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 62236, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 57, "set_filtered": 0, "set_received_from_dcp": 62236, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:19:33.042-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:33.042-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:19:33.042-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:19:33.042-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:33.042-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:33.139-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:33.139-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:33.178-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:19:33.391-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:33.406-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:19:34.200-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:19:34.227-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:19:34.427-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:34.453-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
PipelineManager 2017-04-25T13:19:34.468-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:32.762733919-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]}
]
ReplicationManager 2017-04-25T13:19:34.619-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:19:35.033-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:19:35.061-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:19:35.237-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:35.237-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:19:35.237-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:19:35.237-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:35.237-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:35.303-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:35.303-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:35.317-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:19:36.160-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:19:36.160-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:19:36.160-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:19:36.160-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:36.160-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:36.256-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:36.256-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:36.270-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:19:36.369-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:36.400-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:19:37.129-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:37.140-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:19:37.430-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:37.456-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:19:38.915-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:38.915-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:19:38.915-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:19:38.915-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:38.915-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:38.926-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:38.962-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:19:39.042-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:39.042-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:39.071-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:19:39.828-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:39.828-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:19:39.828-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:19:39.828-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:39.828-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:39.929-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:39.929-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:39.941-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
GenericSupervisor 2017-04-25T13:19:39.964-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:19:39.964-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:19:39.964-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:19:39.997-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:40.008-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
ToplogyChangeDetector 2017-04-25T13:19:40.222-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:19:40.578-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:19:40.589-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:19:40.755-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:40.766-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:19:41.599-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:19:41.610-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:19:41.962-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:41.962-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:19:41.962-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:19:41.962-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:41.962-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:42.059-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:42.059-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:42.070-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:19:42.753-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:19:42.753-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:19:42.753-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:19:42.753-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:42.753-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:19:42.763-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries 
PipelineManager 2017-04-25T13:19:42.763-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:19:42.763-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
PipelineRuntimeCtx 2017-04-25T13:19:42.763-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:19:42.763-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:42.763-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:19:42.763-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:42.76328382 -0500 CDT
GenericSupervisor 2017-04-25T13:19:42.763-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:42.763-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:19:42.763-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:42.763-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:19:42.763-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:42.763-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:19:42.763-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:19:42.763-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:19:42.763-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:19:42.763-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-867973301 has been stopped
AsyncListener 2017-04-25T13:19:42.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-867973301 is stopped
PipelineManager 2017-04-25T13:19:42.763-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
PipelineManager 2017-04-25T13:19:42.763-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:32.762733919-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-867973301
PipelineManager 2017-04-25T13:19:42.763-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:19:42.763-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:19:42.763-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:42.763-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:19:42.763-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:19:42.764-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:19:42.764-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:42.764-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:42.764149892 -0500 CDT
DcpNozzle 2017-04-25T13:19:42.764-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:42.764378958 -0500 CDT
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:42.764-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:42.765-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:19:42.765-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:42.765-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:19:42.765-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:19:42.765-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:42.765-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:19:42.766-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:42.766-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:42.766043864 -0500 CDT
DcpNozzle 2017-04-25T13:19:42.766-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:19:42.766-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:42.766-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:19:42.766-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:42.766-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:42.766292473 -0500 CDT
DcpNozzle 2017-04-25T13:19:42.766-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:19:42.766-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:42.766-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:19:42 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:19:42 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:19:42 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:19:42 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:19:42 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:19:42 Stream Closed for vb 4 StreamEnd simulated
CapiNozzle 2017-04-25T13:19:42.827-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:19:42.828-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:19:42.828-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:42.828-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:19:42.874-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:42.874-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:19:42.874-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:19:42.975-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:42.975-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:19:42.975-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:42.975-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:19:42.975-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:42.975-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:19:42.975-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:19:42.976-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:19:42.976-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
GenericSupervisor 2017-04-25T13:19:42.978-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:19:42.978-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
GenericSupervisor 2017-04-25T13:19:42.978-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:42.978-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:42.978-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:42.978-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:42.978-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:19:42.978-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
CheckpointManager 2017-04-25T13:19:42.978-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
CapiUtils 2017-04-25T13:19:43.021-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:43.021-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:43.021-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:19:43.022-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
StatisticsManager 2017-04-25T13:19:43.022-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:19:43.022-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
XDCRFactory 2017-04-25T13:19:43.022-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:19:43.023-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393 is constructed, start it
GenericPipeline 2017-04-25T13:19:43.024-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[238 247 321 231 264 220 229 184 315 178 194 316 258 268 281 187 203 216 217 237 280 214 225 195 250 293 312 174 200 211 219 213 282 301 208 270 244 324 193 241 171 209 242 307 221 323 319 325 271 306 177 272 288 292 215 304 305 329 206 226 269 303 255 328 296 297 192 254 284 299 309 322 336 223 172 182 197 338 204 274 283 313 332 285 275 317 267 300 185 212 256 295 320 198 248 340 181 230 240 326 173 175 176 311 191 262 276 279 201 233 239 289 222 190 227 207 310 186 333 188 199 286 308 232 298 224 265 210 236 261 228 196 218 202 273 277 327 179 251 253 339 205 259 287 318 330 331 266 291 294 180 335 245 246 257 235 302 189 234 290 334 278 263 314 337 260 243 249 183 252] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[9 61 162 169 11 19 55 128 0 3 24 40 41 58 108 164 170 46 70 77 100 155 87 103 126 141 143 62 145 152 13 63 25 89 112 115 4 31 90 18 67 120 136 140 69 137 37 42 158 54 134 76 80 95 165 35 149 166 57 111 151 117 139 150 5 6 78 144 81 1 84 116 159 60 83 124 127 148 53 105 30 50 114 125 135 157 142 72 86 91 94 44 68 85 20 29 49 74 75 167 110 113 8 47 48 73 96 28 129 97 168 43 64 79 88 21 156 12 59 109 121 38 147 7 15 102 106 132 32 33 34 101 123 146 153 36 138 154 163 10 104 16 22 131 92 119 160 52 71 98 130 2 39 51 66 82 133 161 23 26 65 93 118 45 56 107 122 14 17 27 99]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[183 252 238 247 321 231 264 220 229 184 315 178 194 316 258 268 281 187 203 216 217 237 280 214 225 195 250 293 312 174 200 211 219 213 282 301 208 270 244 324 193 241 171 209 242 307 221 323 319 325 271 306 177 272 288 292 215 304 305 329 206 226 269 303 255 328 296 297 192 254 284 299 309 322 336 223 172 182 197 338 204 274 283 313 332 285 275 317 267 300 185 212 256 295 320 198 248 340 181 230 240 326 173 175 176 311 191 262 276 279 201 233 239 289 222 190 227 207 310 186 333 188 199 286 308 232 298 224 265 210 236 261 228 196 218 202 273 277 327 179 251 253 339 205 259 287 318 330 331 266 291 294 180 335 245 246 257 235 302 189 234 290 334 278 263 314 337 260 243 249] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[9 61 162 169 11 19 55 128 0 3 24 40 41 58 108 164 170 46 70 77 100 155 87 103 126 141 143 62 145 152 13 63 25 89 112 115 4 31 90 18 67 120 136 140 69 137 37 42 158 54 134 76 80 95 165 35 149 166 57 111 151 117 139 150 5 6 78 144 81 1 84 116 159 60 83 124 127 148 53 105 30 50 114 125 135 157 142 72 86 91 94 44 68 85 20 29 49 74 75 167 110 113 8 47 48 73 96 28 129 97 168 43 64 79 88 21 156 12 59 109 121 38 147 7 15 102 106 132 32 33 34 101 123 146 153 36 138 154 163 10 104 16 22 131 92 119 160 52 71 98 130 2 39 51 66 82 133 161 23 26 65 93 118 45 56 107 122 14 17 27 99]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 replication_type:capi optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:19:43.024-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393...
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:19:43.042-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:43.043-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:19:43.043-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:19:43.043-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:19:43.043-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:43.043-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:19:43.043-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:19:43.043-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:19:43.043-05:00 [INFO] Checkpointing starts in 1802.727 sec
PipelineRuntimeCtx 2017-04-25T13:19:43.043-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:19:43.043-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:19:43.043-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:19:43.043-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:19:43.045-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:19:43.045-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:19:43.049-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:19:43.084-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:19:43.085-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:19:43.085-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:19:43.102-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:19:43.102-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:19:43.106-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:43.106-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:43.106-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:19:43.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:19:43.126-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:19:43.148-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:19:43.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:19:43.157-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:43.157-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:19:43.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:19:43.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:19:43.158-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:19:43.193-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:19:43.193-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:19:43.193-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:19:43.225-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:19:43.225-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:19:43.273-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:19:43.487-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:19:43.487-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:43.487-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:19:43.487-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:19:43.487-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:19:43.487-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:19:43.488-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393 is started----------
PipelineManager 2017-04-25T13:19:43.488-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:19:43.488-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:19:43.488-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:19:43.821-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:19:43.821-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:19:43.822-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:19:43.822-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:19:43 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:19:43.921-05:00 [INFO] Send batch count=6
CapiNozzle 2017-04-25T13:19:43.949-05:00 [ERROR] Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f101/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:19:43.949-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f101/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:19:43.956-05:00 [INFO] Send batch count=64
CapiNozzle 2017-04-25T13:19:43.968-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:19:43 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:19:43 UPR_STREAMREQ for vb 103 successful
CapiNozzle 2017-04-25T13:19:44.000-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:44.001-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:19:44 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:19:44.034-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:19:44 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 111 successful
StatisticsManager 2017-04-25T13:19:44.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:19:44 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 247 successful
CapiNozzle 2017-04-25T13:19:44.538-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:44.557-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:19:44 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 258 successful
CapiNozzle 2017-04-25T13:19:44.634-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:44.666-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:19:44 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 170 successful
CapiNozzle 2017-04-25T13:19:44.794-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:44.813-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:19:44 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 82 successful
CapiNozzle 2017-04-25T13:19:44.879-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:44.904-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:19:44 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:19:44 UPR_STREAMREQ for vb 198 successful
CheckpointManager 2017-04-25T13:19:45.037-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
CheckpointManager 2017-04-25T13:19:45.037-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:19:45.051-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:45.051-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:19:45.051-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:19:45.051-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:19:45.052-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:19:45 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:19:45 UPR_STREAMREQ for vb 98 successful
PipelineManager 2017-04-25T13:19:45.078-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:19:45.078-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:19:45.081-05:00 [INFO] err_list=[{"time":"2017-04-25T13:19:45.081436826-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:19:45.081-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:19:45 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:19:45 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:19:45 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:19:45 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:19:45 UPR_STREAMREQ for vb 297 successful
StatisticsManager 2017-04-25T13:19:45.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
DcpNozzle 2017-04-25T13:19:45.176-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:19:45.176-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:45.176-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:45.189-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
DcpNozzle 2017-04-25T13:19:45.194-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:19:45.261-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:45.261-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:19:45.261-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:19:45.261-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:45.261-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:45.324-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:45.324-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:45.345-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:19:45.761-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:45.774-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:19:45.957-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:45.976-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:19:46.057-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:19:46.057-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393 {"Errors": "[{\"time\":\"2017-04-25T13:19:45.081436826-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 32, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 47700, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 32, "set_filtered": 0, "set_received_from_dcp": 47700, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:19:46.626-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:46.650-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:19:46.800-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:46.831-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:19:47.479-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:19:47.479-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:19:47.479-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:19:47.479-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:47.479-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:47.582-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:47.582-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:47.608-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:19:47.749-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:47.749-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:19:47.749-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:19:47.749-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:47.749-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:47.855-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:47.855-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:47.896-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:19:48.307-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:48.331-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:19:48.481-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:48.496-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:19:49.013-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:49.033-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:19:49.171-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:49.189-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
PipelineManager 2017-04-25T13:19:49.635-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:45.081436826-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:19:49.775-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:19:49.775-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:19:49.775-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:19:49.775-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:49.775-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:49.926-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:49.926-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:49.942-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:19:50.017-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:19:50.017-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:19:50.017-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:19:50.017-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:50.017-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:50.154-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:50.154-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:50.189-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:19:50.884-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:50.903-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:19:51.151-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:19:51.176-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:19:52.063-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:52.081-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:19:52.293-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:19:52.305-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
GenericSupervisor 2017-04-25T13:19:53.045-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:19:53.045-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:19:53.045-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:19:53.261-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:19:53.261-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:19:53.261-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:19:53.261-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:53.261-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:53.360-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:53.360-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:53.379-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
ToplogyChangeDetector 2017-04-25T13:19:53.539-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:19:53.583-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:19:53.583-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:19:53.583-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:19:53.583-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:53.583-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:53.684-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:53.685-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:53.707-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:19:54.394-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:19:54.405-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:19:54.630-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:19:54.642-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
PipelineManager 2017-04-25T13:19:55.081-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:19:55.081-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:19:55.081-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:55.081-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
PipelineRuntimeCtx 2017-04-25T13:19:55.081-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:19:55.081-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:55.081-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:19:55.082-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:55.081946576 -0500 CDT
GenericSupervisor 2017-04-25T13:19:55.082-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:55.082-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:19:55.082-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:55.082-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:19:55.082-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:19:55.082-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:19:55.082-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:19:55.082-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:19:55.082-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:19:55.082-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-975959393 has been stopped
GenericPipeline 2017-04-25T13:19:55.082-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-975959393 is stopped
PipelineManager 2017-04-25T13:19:55.082-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:19:55.082-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:45.081436826-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:19:55.082-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:19:55.082-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
GenericPipeline 2017-04-25T13:19:55.082-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
GenericPipeline 2017-04-25T13:19:55.082-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
GenericPipeline 2017-04-25T13:19:55.082-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-975959393
AsyncListener 2017-04-25T13:19:55.082-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:55.082-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:19:55.083-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:19:55.083-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:55.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:19:55.083-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:19:55.083-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:55.083-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:19:55.083-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:19:55.083-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:55.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:19:55.083-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:19:55.083-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:19:55.083-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:19:55.084-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:19:55.084-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:19:55.084-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:19:55.084-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:55.083786699 -0500 CDT
DcpNozzle 2017-04-25T13:19:55.084-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:19:55.084-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:55.08401091 -0500 CDT
CapiNozzle 2017-04-25T13:19:55.084-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:19:55.084-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:55.085-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:19:55.085-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:19:55.085-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:19:55.085-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:19:55.085-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:19:55.085-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:55.085-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:19:55.086-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:55.086-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:19:55 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:19:55.092-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:19:55.092-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:55.092478119 -0500 CDT
DcpNozzle 2017-04-25T13:19:55.092-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:19:55.092-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:55.092-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:19:55 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:19:55.094-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:19:55 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:19:55 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:19:55 Stream Closed for vb 174 StreamEnd simulated
DcpNozzle 2017-04-25T13:19:55.094-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:19:55.094481608 -0500 CDT
DcpNozzle 2017-04-25T13:19:55.094-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:19:55.094-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:19:55.094-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:19:55 Stream Closed for vb 175 StreamEnd simulated
2017/04/25 13:19:55 Stream Closed for vb 176 StreamEnd simulated
2017/04/25 13:19:55 Stream Closed for vb 177 StreamEnd simulated
XDCRFactory 2017-04-25T13:19:55.187-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:55.188-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:19:55.188-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:19:55.289-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:19:55.289-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:19:55.289-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:55.290-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:19:55.290-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:19:55.290-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:19:55.290-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:19:55.290-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:19:55.290-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
GenericSupervisor 2017-04-25T13:19:55.292-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:19:55.292-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
GenericSupervisor 2017-04-25T13:19:55.292-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:55.292-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:55.292-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:55.292-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:19:55.292-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:19:55.292-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
CheckpointManager 2017-04-25T13:19:55.292-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
CapiUtils 2017-04-25T13:19:55.355-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:55.355-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:55.355-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:19:55.356-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
StatisticsManager 2017-04-25T13:19:55.356-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:19:55.356-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
XDCRFactory 2017-04-25T13:19:55.356-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:19:55.356-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340 is constructed, start it
GenericPipeline 2017-04-25T13:19:55.357-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[201 212 251 286 173 194 198 276 296 310 332 248 267 221 266 270 338 211 215 301 337 241 232 235 199 225 182 193 239 290 293 305 316 318 340 278 183 261 268 237 238 222 284 174 189 206 219 281 300 314 307 252 306 260 191 192 327 264 326 190 227 244 288 178 180 322 335 294 172 185 209 274 277 216 234 285 262 256 179 247 213 283 302 231 233 245 297 324 186 303 176 220 272 308 200 218 184 289 295 311 202 223 224 250 259 263 204 280 181 187 304 313 246 258 217 236 243 257 188 292 265 273 336 279 249 269 333 175 197 208 317 312 315 196 214 275 207 271 299 330 205 309 320 321 177 253 240 298 328 242 331 210 228 254 323 195 203 329 229 171 226 230 334 255 282 287 291 319 325 339] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[92 169 60 82 159 23 49 95 128 99 16 31 79 32 154 22 165 54 97 135 156 168 63 72 108 152 29 101 122 153 48 85 113 164 34 37 38 45 88 2 12 27 64 127 136 160 0 6 11 86 141 78 114 3 129 10 15 98 149 170 28 107 131 151 43 109 115 138 57 58 133 46 61 84 117 13 26 44 65 158 25 76 119 143 5 30 71 112 124 166 9 66 91 20 67 21 52 93 163 74 167 42 116 130 90 110 137 161 8 103 75 80 139 162 51 59 144 56 147 36 39 126 96 19 102 4 68 157 83 155 118 121 41 70 33 40 55 145 146 150 47 50 105 123 1 7 24 140 81 148 14 17 100 125 120 18 77 104 111 73 87 106 134 62 69 89 94 35 53 132 142]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[78 114 3 129 10 15 98 149 170 28 107 131 151 43 109 115 138 57 58 133 46 61 13 26 44 65 84 117 25 76 119 143 158 5 30 71 112 124 166 9 66 91 20 67 21 52 93 163 74 167 42 116 130 90 110 137 161 8 103 75 80 139 162 51 59 144 56 147 36 39 126 96 19 102 4 68 157 83 155 118 121 41 70 33 40 55 145 146 47 50 105 123 150 1 7 24 140 81 148 14 17 100 125 18 77 104 111 120 73 87 106 134 62 69 89 94 35 53 132 142 92 169 60 82 159 23 49 95 128 99 16 31 79 32 154 22 165 54 97 135 63 72 108 152 156 168 29 101 122 153 164 34 37 38 45 48 85 113 2 12 27 64 88 127 136 160 0 6 11 86 141] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[252 306 307 191 192 260 264 326 327 288 178 180 190 227 244 322 335 172 185 209 294 274 277 216 234 285 256 262 179 247 283 213 302 245 231 233 297 186 303 324 176 220 272 308 184 200 218 202 223 289 295 311 224 250 259 263 181 187 204 280 246 258 304 313 257 188 217 236 243 292 265 273 336 279 175 197 249 269 333 208 317 196 214 275 312 315 207 205 271 299 330 309 320 321 177 253 240 298 328 242 331 254 210 228 323 195 203 329 229 334 171 226 230 255 282 287 291 319 325 339 286 173 194 201 212 251 198 276 296 310 332 248 221 266 267 211 270 338 301 215 337 241 199 225 232 235 239 182 193 290 293 340 278 305 316 318 183 261 268 237 238 174 189 206 222 284 314 219 281 300]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi active:true checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:19:55.357-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340...
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:19:55.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:19:55.371-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:19:55.371-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:19:55.371-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:19:55.371-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:19:55.371-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:19:55.371-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:19:55.371-05:00 [INFO] Checkpointing starts in 1800.391 sec
PipelineRuntimeCtx 2017-04-25T13:19:55.371-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:19:55.371-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:19:55.372-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:19:55.372-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:19:55.372-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:19:55.372-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:19:55.375-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:19:55.393-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:19:55.394-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:19:55.394-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:19:55.415-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:19:55.415-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:19:55.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:19:55.444-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:19:55.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:19:55.452-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:55.452-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:19:55.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:19:55.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:19:55.452-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:19:55.477-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:19:55.477-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:19:55.477-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:19:55.499-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:19:55.499-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:19:55.499-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:19:55.520-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:19:55.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:19:55.538-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:19:55.668-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:19:55.668-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:19:55.668-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
CheckpointManager 2017-04-25T13:19:55.671-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:19:55.671-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:19:55.671-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:19:55.672-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:19:55.672-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340 is started----------
PipelineManager 2017-04-25T13:19:55.672-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:19:55.672-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:19:55.672-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:19:56.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:19:56.583-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:19:56.583-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:19:56.583-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:19:56.583-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:19:56 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 205 successful
CapiNozzle 2017-04-25T13:19:56.705-05:00 [INFO] Send batch count=94
CapiNozzle 2017-04-25T13:19:56.725-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:56.725-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:56.744-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
2017/04/25 13:19:56 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:19:56.813-05:00 [INFO] Send batch count=107
2017/04/25 13:19:56 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:19:56.849-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:56.849-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:19:56 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 13 successful
CapiNozzle 2017-04-25T13:19:56.899-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
2017/04/25 13:19:56 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 114 successful
CapiNozzle 2017-04-25T13:19:56.977-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:19:56 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:19:56 UPR_STREAMREQ for vb 316 successful
CapiNozzle 2017-04-25T13:19:56.997-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
2017/04/25 13:19:57 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 127 successful
CapiNozzle 2017-04-25T13:19:57.131-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:19:57.163-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
2017/04/25 13:19:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 236 successful
CapiNozzle 2017-04-25T13:19:57.275-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:19:57 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:19:57.302-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
2017/04/25 13:19:57 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 261 successful
CapiNozzle 2017-04-25T13:19:57.407-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
StatisticsManager 2017-04-25T13:19:57.428-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:19:57.450-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
2017/04/25 13:19:57 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 273 successful
CapiNozzle 2017-04-25T13:19:57.617-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:19:57.617-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:19:57.617-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:19:57.617-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]
ReplicationManager 2017-04-25T13:19:57.617-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]
PipelineManager 2017-04-25T13:19:57.624-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:19:57.624-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:19:57.627-05:00 [INFO] err_list=[{"time":"2017-04-25T13:19:57.627104578-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries"}]
PipelineManager 2017-04-25T13:19:57.627-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:19:57 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 178 successful
CapiNozzle 2017-04-25T13:19:57.699-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:57.699-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:57.721-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:19:57 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:19:57 UPR_STREAMREQ for vb 190 successful
CapiNozzle 2017-04-25T13:19:57.887-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:19:57.887-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:19:57.887-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:19:57.887-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:57.888-05:00 [INFO] Send batch count=500
CheckpointManager 2017-04-25T13:19:57.951-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
CheckpointManager 2017-04-25T13:19:57.951-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:19:57.959-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:19:57.960-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:57.960-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:57.975-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:19:58 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:19:58 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:19:58 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:19:58 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:19:58 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:19:58 UPR_STREAMREQ for vb 194 successful
DcpNozzle 2017-04-25T13:19:58.083-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:19:58.322-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:19:58.347-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
StatisticsManager 2017-04-25T13:19:58.373-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:19:58.374-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340 {"Errors": "[{\"time\":\"2017-04-25T13:19:57.627104578-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 17, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 90, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 27114, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 90, "set_filtered": 0, "set_received_from_dcp": 27114, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:19:58.772-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:58.787-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:19:59.100-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:19:59.116-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:19:59.274-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:59.293-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:19:59.564-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:19:59.564-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:19:59.564-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:19:59.564-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:59.564-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:59.638-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:59.638-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:59.652-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:19:59.833-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:19:59.834-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:19:59.834-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:19:59.834-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:19:59.834-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:19:59.956-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:19:59.956-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:19:59.984-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:20:00.964-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:20:00.986-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:20:01.235-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:20:01.261-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:20:01.720-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:20:01.734-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:20:01.918-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:20:01.935-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:20:02.567-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:20:02.567-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:20:02.567-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:20:02.567-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:02.567-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:02.674-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:02.674-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:02.687-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:20:02.848-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:20:02.848-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:20:02.848-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:20:02.848-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:02.848-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:02.931-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:02.931-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:02.947-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:20:03.722-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:03.738-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:20:04.003-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:20:04.025-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
PipelineManager 2017-04-25T13:20:04.372-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:57.627104578-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries]}
]
CapiNozzle 2017-04-25T13:20:04.846-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:04.857-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:20:05.209-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:20:05.220-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
GenericSupervisor 2017-04-25T13:20:05.372-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:20:05.372-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:20:05.372-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:20:05.656-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:20:05.898-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:05.899-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:20:05.899-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:20:05.899-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:05.899-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:05.955-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:05.956-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:05.966-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:20:06.142-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:20:06.142-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:20:06.142-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:20:06.142-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:06.142-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:06.203-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:06.203-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:06.214-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:20:06.633-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:20:06.644-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:20:06.838-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:20:06.849-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:20:07.371-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:20:07.381-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:20:07.518-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:20:07.529-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
PipelineManager 2017-04-25T13:20:07.627-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries 
PipelineManager 2017-04-25T13:20:07.627-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:20:07.627-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:20:07.627-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
PipelineRuntimeCtx 2017-04-25T13:20:07.627-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:20:07.627-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:07.627-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:20:07.627-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:07.627590375 -0500 CDT
GenericSupervisor 2017-04-25T13:20:07.627-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:07.627-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:20:07.627-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:20:07.627-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:20:07.627-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:20:07.627-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:20:07.627-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:20:07.627-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:20:07.627-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:20:07.627-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-290315340 has been stopped
GenericPipeline 2017-04-25T13:20:07.628-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-290315340 is stopped
PipelineManager 2017-04-25T13:20:07.628-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:20:07.628-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:19:57.627104578-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 3 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:20:07.628-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:20:07.628-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
GenericPipeline 2017-04-25T13:20:07.628-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
GenericPipeline 2017-04-25T13:20:07.628-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
GenericPipeline 2017-04-25T13:20:07.628-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-290315340
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:07.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:07.629-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:20:07.629-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:07.629-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:20:07.629-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:07.629-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:20:07.629-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:07.629-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:20:07.629-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:07.629-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:07.629-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:20:07.630-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:07.629508568 -0500 CDT
DcpNozzle 2017-04-25T13:20:07.630-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:20:07.630-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:07.629782987 -0500 CDT
CapiNozzle 2017-04-25T13:20:07.631-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:20:07.631-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:20:07.631-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:20:07.631-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:20:07.631-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:20:07.631-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:20:07.631-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
2017/04/25 13:20:07 Stream Closed for vb 0 StreamEnd simulated
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:20:07.632-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
2017/04/25 13:20:07 Stream Closed for vb 171 StreamEnd simulated
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:20:07.632-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:20:07.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:20:07.632-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:20:07.634-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:20:07.634-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:07.634274984 -0500 CDT
DcpNozzle 2017-04-25T13:20:07.634-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:20:07.634-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:07.634-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:20:07 Stream Closed for vb 1 StreamEnd simulated
DcpNozzle 2017-04-25T13:20:07.667-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:20:07 Stream Closed for vb 2 StreamEnd simulated
DcpNozzle 2017-04-25T13:20:07.667-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:07.667722828 -0500 CDT
DcpNozzle 2017-04-25T13:20:07.667-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:20:07.667-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:07.668-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
XDCRFactory 2017-04-25T13:20:07.735-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:20:07.735-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:20:07.735-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:20:07.828-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:20:07.829-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:20:07.829-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:20:07.829-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:20:07.829-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:20:07.829-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:20:07.829-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:20:07.829-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:20:07.829-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
GenericSupervisor 2017-04-25T13:20:07.830-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:20:07.830-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
GenericSupervisor 2017-04-25T13:20:07.830-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:07.830-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:07.831-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:07.831-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:07.831-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:20:07.831-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
CheckpointManager 2017-04-25T13:20:07.831-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
CapiUtils 2017-04-25T13:20:07.860-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:20:07.860-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:20:07.860-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:20:07.861-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
StatisticsManager 2017-04-25T13:20:07.861-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:20:07.861-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
XDCRFactory 2017-04-25T13:20:07.861-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:20:07.862-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893 is constructed, start it
GenericPipeline 2017-04-25T13:20:07.862-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[89 137 64 75 21 154 71 79 17 33 65 8 78 117 20 112 128 163 5 106 107 111 31 81 146 57 156 165 55 63 102 22 83 90 122 162 68 93 77 120 26 140 147 46 82 118 32 16 123 126 13 119 7 87 152 169 43 76 58 130 91 139 19 142 70 49 51 67 129 6 34 96 116 150 11 100 170 4 29 125 23 62 84 151 18 52 124 166 48 157 131 86 37 105 141 60 159 114 53 25 27 80 121 15 92 148 74 138 14 85 2 10 132 1 30 73 98 135 164 0 113 133 144 40 72 136 145 155 44 3 95 61 149 168 47 45 66 38 127 97 104 158 42 94 99 109 134 28 50 59 103 9 69 110 153 161 108 36 167 24 41 54 101 160 12 88 115 35 56 143 39] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[189 309 228 266 331 183 236 289 298 324 219 231 199 247 268 302 311 220 294 172 195 197 202 203 208 229 241 303 175 312 330 235 269 173 339 217 225 292 308 320 334 226 204 316 321 326 338 201 336 307 299 214 223 253 259 279 283 310 327 297 210 257 301 255 295 234 239 288 305 323 230 252 290 291 181 267 185 216 232 328 184 261 224 271 313 272 176 213 332 276 286 329 206 246 251 282 238 296 263 280 306 233 262 278 178 171 250 335 180 188 194 240 273 319 244 177 190 314 340 193 182 221 300 215 275 207 265 186 211 270 179 200 249 254 304 212 196 284 325 198 227 237 333 218 187 277 318 174 205 245 315 293 274 242 322 317 337 191 258 264 285 287 281 192 222 209 243 248 256 260]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[44 72 136 145 155 3 61 95 47 149 168 38 45 66 127 42 97 104 158 28 94 99 109 134 9 50 59 103 69 108 110 153 161 36 24 167 12 41 54 101 160 35 88 115 39 56 143 64 89 137 21 75 154 17 71 79 8 33 65 20 78 117 5 112 128 163 31 106 107 111 57 81 146 55 156 165 22 63 102 68 83 90 122 162 77 93 26 120 46 140 147 32 82 118 13 16 123 126 7 119 43 87 152 169 58 76 91 130 19 139 70 142 6 49 51 67 129 11 34 96 116 150 4 100 170 29 23 125 18 62 84 151 48 52 124 166 131 157 37 86 60 105 141 114 159 53 15 25 27 80 121 74 92 148 138 2 14 85 1 10 132 0 30 73 98 135 164 40 113 133 144] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[215 275 207 265 186 211 270 179 200 249 254 212 304 196 284 325 333 198 227 237 218 187 277 318 174 205 245 315 293 274 242 322 317 337 264 191 258 281 285 287 192 222 260 209 243 248 256 189 309 228 266 331 183 236 289 298 324 219 231 199 247 268 302 311 220 294 203 172 195 197 202 208 229 241 303 175 312 330 235 269 173 339 320 217 225 292 308 334 226 326 204 316 321 338 201 336 299 307 283 214 223 253 259 279 297 310 327 210 257 301 255 295 234 239 288 305 323 290 230 252 291 181 267 185 216 232 328 184 261 224 271 313 176 272 213 332 276 286 329 206 246 251 282 238 296 263 280 306 178 233 262 278 171 250 335 273 180 188 194 240 319 244 190 177 193 314 340 182 221 300]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 log_level:Info stats_interval:1000 active:true]
CheckpointManager 2017-04-25T13:20:07.863-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893...
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:07.906-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:20:07.906-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:07.906-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:20:07.906-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:20:07.906-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:20:07.906-05:00 [INFO] Checkpointing starts in 1800.894 sec
PipelineRuntimeCtx 2017-04-25T13:20:07.906-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:20:07.906-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:20:07.906-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:20:07.907-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:20:07.907-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:20:07.907-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:20:07.907-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:20:07.908-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:20:07.911-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:20:07.937-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:20:07.937-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:20:07.937-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:20:07.958-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:20:07.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:20:07.963-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:20:07.976-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:20:07.977-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:20:07.977-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:20:07.973-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:20:07.985-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:20:07.989-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:20:07.989-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:20:07.989-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:20:07.989-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:20:07.989-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:20:07.989-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
StatisticsManager 2017-04-25T13:20:10.223-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:20:10.270-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:20:10.303-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:20:10.303-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:20:10.303-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:20:10.321-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:20:10.321-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:20:10.476-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:20:10.476-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:20:10.476-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:20:10.498-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:20:10.498-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:20:10.498-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:20:10.498-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893 is started----------
PipelineManager 2017-04-25T13:20:10.498-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:20:10.498-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:20:10.498-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:20:11.010-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:20:12.025-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:20:12.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:20:13.674-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:20:13.675-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:20:13.675-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:20:13.675-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:20:13 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:20:13.781-05:00 [INFO] Send batch count=38
CapiNozzle 2017-04-25T13:20:13.811-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:13.811-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:20:13 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:20:13.861-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:20:13.868-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:20:13.868-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f302/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:20:13.868-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f302/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:20:13.903-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:20:13 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:20:13 UPR_STREAMREQ for vb 206 successful
StatisticsManager 2017-04-25T13:20:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:20:14 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 26 successful
CapiNozzle 2017-04-25T13:20:14.374-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:14.377-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:20:14 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 226 successful
CapiNozzle 2017-04-25T13:20:14.425-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:20:14 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 143 successful
CapiNozzle 2017-04-25T13:20:14.824-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:20:14 UPR_STREAMREQ for vb 235 successful
CapiNozzle 2017-04-25T13:20:14.864-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:20:14 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 239 successful
StatisticsManager 2017-04-25T13:20:14.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:20:14 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:20:14 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:20:15.172-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:20:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 252 successful
CapiNozzle 2017-04-25T13:20:15.207-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:20:15 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 166 successful
CapiNozzle 2017-04-25T13:20:15.212-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:20:15.212-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:20:15.212-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:20:15.212-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:20:15.212-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:20:15.233-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:20:15.233-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:20:15.237-05:00 [INFO] err_list=[{"time":"2017-04-25T13:20:15.237697354-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:20:15.237-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:20:15 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 263 successful
CapiNozzle 2017-04-25T13:20:15.473-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:15.485-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:20:15 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 75 successful
CapiNozzle 2017-04-25T13:20:15.510-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:20:15 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 96 successful
DcpNozzle 2017-04-25T13:20:15.891-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:20:15 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:20:15 UPR_STREAMREQ for vb 294 successful
CheckpointManager 2017-04-25T13:20:15.922-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
CheckpointManager 2017-04-25T13:20:15.922-05:00 [INFO] Done with SetVBTimestamps
StatisticsManager 2017-04-25T13:20:15.932-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:20:15.932-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893 {"Errors": "[{\"time\":\"2017-04-25T13:20:15.237697354-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 331, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 36520, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 331, "set_filtered": 0, "set_received_from_dcp": 36520, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

2017/04/25 13:20:16 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:20:16 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:20:16 UPR_STREAMREQ for vb 297 successful
CapiNozzle 2017-04-25T13:20:16.049-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:20:16.061-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
DcpNozzle 2017-04-25T13:20:16.105-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:20:16.268-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:20:16.282-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:20:16.971-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:20:16.994-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:20:17.206-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:20:17.222-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:20:17.756-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:20:17.757-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:20:17.757-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:20:17.757-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:17.757-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:17.827-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:17.827-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:17.844-05:00 [INFO] Retrying update docs for vb 205 for the 1th time
GenericSupervisor 2017-04-25T13:20:17.909-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:20:17.909-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:20:17.909-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:20:18.046-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:20:18.046-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:20:18.046-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:20:18.046-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:18.046-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:18.136-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:18.136-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:18.159-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:20:18.686-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:20:18.707-05:00 [INFO] Retrying update docs for vb 205 for the 2th time
CapiNozzle 2017-04-25T13:20:18.884-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:20:18.899-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
PipelineManager 2017-04-25T13:20:19.365-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:15.237697354-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:20:19.532-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:20:19.560-05:00 [INFO] Retrying update docs for vb 205 for the 3th time
CapiNozzle 2017-04-25T13:20:19.748-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:20:19.764-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
ToplogyChangeDetector 2017-04-25T13:20:20.288-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:20:20.427-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:20:20.427-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 205 after 3 retries
CapiNozzle 2017-04-25T13:20:20.427-05:00 [ERROR] Raise error condition batch update docs failed for vb 205 after 3 retries
GenericSupervisor 2017-04-25T13:20:20.427-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:20.427-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:20.542-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:20.542-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:20.560-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:20:20.698-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:20:20.698-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:20:20.698-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:20:20.698-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:20.698-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:20.774-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:20.774-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:20.786-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:20:20.912-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:20:20.931-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:20:21.073-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:20:21.094-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:20:21.304-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:20:21.318-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:20:21.473-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:20:21.491-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:20:21.697-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:20:21.697-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:20:21.697-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:20:21.697-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:21.697-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:21.781-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:21.781-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:21.791-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:20:22.146-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:20:22.146-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:20:22.146-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:20:22.146-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:22.146-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:22.208-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:22.209-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:22.219-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:20:22.476-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:20:22.487-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:20:22.845-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:22.856-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:20:23.079-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:20:23.090-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:20:23.518-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:23.529-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:20:23.871-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:20:23.871-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:20:23.871-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:20:23.871-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:23.871-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:23.947-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:23.947-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:23.958-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:20:24.472-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:24.472-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:20:24.472-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:20:24.472-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:24.472-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:24.560-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:24.560-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:24.571-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:20:24.723-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:20:24.734-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
PipelineManager 2017-04-25T13:20:25.237-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:20:25.238-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:20:25.238-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
PipelineRuntimeCtx 2017-04-25T13:20:25.238-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:20:25.238-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:25.238-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:20:25.238-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:25.238159399 -0500 CDT
GenericSupervisor 2017-04-25T13:20:25.238-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:25.238-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:20:25.238-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:20:25.238-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:20:25.238-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:20:25.238-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:20:25.238-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:20:25.238-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:20:25.238-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:20:25.238-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-829427893 has been stopped
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-829427893 is stopped
PipelineManager 2017-04-25T13:20:25.238-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
PipelineManager 2017-04-25T13:20:25.238-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:15.237697354-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:20:25.238-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-829427893
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:20:25.238-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:20:25.238-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:20:25.238-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:20:25.238-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:25.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:25.238-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:25.239-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:25.239-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:25.239-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:20:25.239-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:20:25.239-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:25.23893497 -0500 CDT
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:20:25.239-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:20:25.239-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:20:25.239-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:25.239139986 -0500 CDT
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:20:25.239-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:20:25.239-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
DcpNozzle 2017-04-25T13:20:25.240-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:20:25.240-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:25.240345254 -0500 CDT
DcpNozzle 2017-04-25T13:20:25.240-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:20:25.240-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:25.240-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:20:25.240-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:25.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:20:25.240-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:20:25.240-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:20:25.240-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:20:25.240-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:20:25.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:20:25.240-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:20:25.240-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:25.240-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:20:25.240-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:25.240-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:20:25.241-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:20:25.241-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:25.24132915 -0500 CDT
DcpNozzle 2017-04-25T13:20:25.241-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:20:25.241-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:25.241-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:20:25 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:20:25 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:20:25 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:20:25 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:20:25 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:20:25 Stream Closed for vb 173 StreamEnd simulated
XDCRFactory 2017-04-25T13:20:25.320-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:20:25.321-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:20:25.321-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:20:25.488-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:20:25.489-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:20:25.489-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:20:25.489-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:20:25.489-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:20:25.489-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:20:25.489-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:20:25.490-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:20:25.490-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
GenericSupervisor 2017-04-25T13:20:25.492-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:20:25.492-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
GenericSupervisor 2017-04-25T13:20:25.492-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:25.492-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:25.492-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:25.492-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:25.492-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:20:25.492-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
CheckpointManager 2017-04-25T13:20:25.492-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
CapiUtils 2017-04-25T13:20:25.587-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:20:25.587-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:20:25.587-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:20:25.588-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
StatisticsManager 2017-04-25T13:20:25.589-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:20:25.589-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
XDCRFactory 2017-04-25T13:20:25.589-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:20:25.589-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894 is constructed, start it
GenericPipeline 2017-04-25T13:20:25.590-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[40 82 94 95 122 137 17 35 0 105 71 125 151 39 83 127 56 159 81 24 123 169 46 93 106 111 79 102 103 156 11 36 144 165 136 20 72 109 121 1 76 18 62 66 100 128 150 161 10 104 140 162 44 14 6 31 37 115 146 147 148 3 74 90 48 70 42 80 119 139 152 12 167 130 124 138 154 15 50 55 92 34 21 88 4 53 30 89 168 5 86 78 143 67 84 132 52 118 131 38 129 149 166 9 8 101 120 170 33 116 145 160 99 43 85 117 32 41 49 108 163 23 73 75 157 7 60 142 29 134 141 164 133 26 97 126 25 69 87 158 13 27 22 114 2 65 28 54 96 91 107 153 45 68 61 58 57 63 113 16 110 19 47 77 98 64 59 112 135 155 51] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[207 251 255 314 232 299 274 283 238 271 187 211 224 226 262 305 339 177 208 261 307 200 310 290 303 172 229 231 234 329 209 270 284 178 322 244 268 201 194 204 210 212 318 181 228 230 253 316 332 333 206 236 336 214 176 186 188 205 304 317 195 225 281 321 275 325 196 293 300 338 180 193 242 218 237 260 282 171 202 330 199 313 182 247 249 312 248 233 311 326 198 287 289 301 337 189 272 254 319 306 192 296 327 335 257 315 320 179 215 259 173 219 235 174 292 250 266 221 240 216 222 227 243 278 239 295 184 203 277 288 291 298 252 263 276 323 302 334 264 285 213 280 309 258 286 297 223 273 294 185 308 331 328 191 217 241 267 269 183 197 245 256 279 324 175 190 220 246 340 265]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[246 340 190 220 265 207 251 314 255 299 232 274 283 238 271 224 226 262 187 211 339 305 177 208 261 307 200 310 290 303 172 229 231 234 329 209 270 284 178 322 201 244 268 194 204 210 212 318 228 230 253 316 332 181 333 206 214 236 336 176 186 188 205 304 317 225 281 321 195 275 325 293 300 338 196 180 193 242 218 237 260 282 171 202 199 330 313 249 312 182 247 248 311 326 233 198 287 337 289 301 189 272 254 319 306 327 335 192 296 257 315 320 179 215 259 173 219 235 174 292 221 250 266 240 216 222 227 243 278 239 295 277 288 291 298 184 203 252 263 276 323 302 334 285 264 309 213 280 286 297 258 223 273 294 308 331 185 328 217 241 267 269 191 245 256 279 324 183 197 175] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[59 155 51 112 135 94 95 122 137 17 40 82 0 35 71 105 39 125 151 56 83 127 159 24 81 46 123 169 111 79 93 106 156 11 102 103 36 136 144 165 20 121 1 72 109 76 66 100 128 150 161 10 18 62 162 44 104 140 6 14 115 146 147 148 3 31 37 48 74 90 42 70 139 152 12 80 119 130 167 154 15 124 138 92 34 50 55 4 21 88 53 168 5 30 89 78 86 143 67 52 84 132 38 118 131 166 9 129 149 8 170 33 101 120 160 99 116 145 117 32 43 85 108 163 23 41 49 157 7 73 75 29 60 142 164 133 134 141 26 25 97 126 69 13 87 158 27 2 22 114 28 65 54 91 96 45 107 153 68 58 61 113 16 57 63 19 110 47 64 77 98]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true doc_batch_size_kb:2048 worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi filter_expression: checkpoint_interval:1800 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:20:25.590-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894...
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:25.621-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:20:25.621-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:25.621-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:20:25.621-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:20:25.621-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:20:25.621-05:00 [INFO] Checkpointing starts in 1801.345 sec
PipelineRuntimeCtx 2017-04-25T13:20:25.621-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:20:25.621-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:20:25.622-05:00 [INFO] checkpointing rountine started
GenericSupervisor 2017-04-25T13:20:25.621-05:00 [INFO] monitorPipelineHealth started
PipelineRuntimeCtx 2017-04-25T13:20:25.622-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:20:25.623-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:20:25.623-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:20:25.623-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:20:25.623-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:20:25.647-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:20:25.647-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:20:25.647-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:20:25.664-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:20:25.664-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:20:25.691-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:20:25.701-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:20:25.701-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:20:25.701-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:20:25.702-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:20:25.702-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:20:25.702-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
ToplogyChangeDetector 2017-04-25T13:20:25.726-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:20:25.735-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:20:25.735-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:20:25.735-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:20:25.747-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:20:25.747-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CapiUtils 2017-04-25T13:20:25.777-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:20:25.777-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:20:25.777-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:20:25.815-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:20:25.877-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:20:25.877-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:20:25.877-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:20:25.877-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:20:25.877-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:20:25.878-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:20:25.878-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894 is started----------
PipelineManager 2017-04-25T13:20:25.878-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:20:25.878-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:20:25.878-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:20:25.961-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:20:26.424-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:20:26.424-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:20:26.424-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:20:26.425-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:20:26 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:20:26.532-05:00 [INFO] Send batch count=177
2017/04/25 13:20:26 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:20:26.594-05:00 [INFO] Send batch count=91
CapiNozzle 2017-04-25T13:20:26.596-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:26.596-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:26.630-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
2017/04/25 13:20:26 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 10 successful
CapiNozzle 2017-04-25T13:20:26.661-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:26.661-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:20:26.698-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:20:26.703-05:00 [INFO] Retrying update docs for vb 106 for the 1th time
2017/04/25 13:20:26 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 120 successful
CapiNozzle 2017-04-25T13:20:26.873-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:20:26.887-05:00 [INFO] Retrying update docs for vb 106 for the 2th time
2017/04/25 13:20:26 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:20:26 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 139 successful
CapiNozzle 2017-04-25T13:20:27.090-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
2017/04/25 13:20:27 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 239 successful
CapiNozzle 2017-04-25T13:20:27.136-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:20:27.144-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
2017/04/25 13:20:27 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 148 successful
CapiNozzle 2017-04-25T13:20:27.182-05:00 [INFO] Retrying update docs for vb 106 for the 3th time
2017/04/25 13:20:27 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 60 successful
CapiNozzle 2017-04-25T13:20:27.372-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:20:27.372-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 106 after 3 retries
CapiNozzle 2017-04-25T13:20:27.372-05:00 [ERROR] Raise error condition batch update docs failed for vb 106 after 3 retries
GenericSupervisor 2017-04-25T13:20:27.372-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries]
ReplicationManager 2017-04-25T13:20:27.372-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries]
PipelineManager 2017-04-25T13:20:27.385-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:20:27.385-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:20:27.388-05:00 [INFO] err_list=[{"time":"2017-04-25T13:20:27.388285422-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries"}]
PipelineManager 2017-04-25T13:20:27.388-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:20:27 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 67 successful
CapiNozzle 2017-04-25T13:20:27.482-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:27.482-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:20:27 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 175 successful
CapiNozzle 2017-04-25T13:20:27.503-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:20:27 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 80 successful
CapiNozzle 2017-04-25T13:20:27.594-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
2017/04/25 13:20:27 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 273 successful
CapiNozzle 2017-04-25T13:20:27.611-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
2017/04/25 13:20:27 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 84 successful
StatisticsManager 2017-04-25T13:20:27.636-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:20:27.636-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894 {"Errors": "[{\"time\":\"2017-04-25T13:20:27.388285422-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 62, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 2228, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 62, "set_filtered": 0, "set_received_from_dcp": 2228, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries]", "Status": "Pending"}

2017/04/25 13:20:27 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:20:27 UPR_STREAMREQ for vb 97 successful
CapiNozzle 2017-04-25T13:20:27.970-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CheckpointManager 2017-04-25T13:20:28.010-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
CheckpointManager 2017-04-25T13:20:28.010-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:20:28.014-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:20:28 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:20:28 UPR_STREAMREQ for vb 290 successful
DcpNozzle 2017-04-25T13:20:28.032-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:20:28.098-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:20:28.129-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:20:28.129-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:20:28.129-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:20:28.129-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:28.129-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:28.189-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:28.189-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:28.208-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:20:28.851-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:20:28.869-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:20:28.985-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:20:28.999-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:20:29.483-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:20:29.483-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:20:29.483-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:20:29.483-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:29.483-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:29.548-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:29.548-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:29.561-05:00 [INFO] Retrying update docs for vb 106 for the 1th time
CapiNozzle 2017-04-25T13:20:29.597-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:20:29.632-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:20:30.057-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:20:30.081-05:00 [INFO] Retrying update docs for vb 106 for the 2th time
CapiNozzle 2017-04-25T13:20:30.192-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:20:30.192-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:20:30.192-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:20:30.192-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:30.192-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:30.290-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:30.290-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:30.309-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:20:30.759-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:20:30.777-05:00 [INFO] Retrying update docs for vb 106 for the 3th time
CapiNozzle 2017-04-25T13:20:31.032-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:20:31.058-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:20:31.573-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:20:31.573-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 106 after 3 retries
CapiNozzle 2017-04-25T13:20:31.573-05:00 [ERROR] Raise error condition batch update docs failed for vb 106 after 3 retries
GenericSupervisor 2017-04-25T13:20:31.573-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:31.573-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:31.677-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:31.677-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:31.692-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:20:31.853-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:20:31.870-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:20:32.430-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:20:32.518-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:20:33.071-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:20:33.071-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:20:33.071-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:20:33.071-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:33.071-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:33.222-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:20:33.238-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:20:33.246-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:33.271-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:33.303-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:20:33.731-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:20:33.731-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:20:33.731-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:20:33.731-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:33.731-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:33.810-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:33.810-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:33.829-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:20:34.133-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:20:34.156-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:20:34.322-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:34.335-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:20:34.672-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:20:34.685-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:20:34.860-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
PipelineManager 2017-04-25T13:20:34.841-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:27.388285422-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries]}
]
CapiNozzle 2017-04-25T13:20:34.884-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:20:35.303-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:20:35.303-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:20:35.303-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:20:35.303-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:35.303-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:35.433-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:35.434-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:35.444-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
ReplicationManager 2017-04-25T13:20:35.474-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:20:35.485-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:20:35.485-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:20:35.485-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:20:35.485-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:35.485-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:35.584-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:35.584-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:35.597-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
GenericSupervisor 2017-04-25T13:20:35.636-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:20:35.636-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:20:35.636-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:20:35.964-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:20:36.662-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:20:36.718-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:20:36.826-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:20:36.839-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
CapiNozzle 2017-04-25T13:20:37.159-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:20:37.172-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
PipelineManager 2017-04-25T13:20:37.402-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries 
PipelineManager 2017-04-25T13:20:37.410-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:20:37.410-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:20:37.410-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
PipelineRuntimeCtx 2017-04-25T13:20:37.410-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:20:37.410-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:37.410-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:20:37.411-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:37.410727709 -0500 CDT
GenericSupervisor 2017-04-25T13:20:37.421-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:37.421-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:20:37.421-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:20:37.421-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:20:37.421-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:20:37.421-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:20:37.421-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:20:37.421-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:20:37.421-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:20:37.421-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-489679894 has been stopped
GenericPipeline 2017-04-25T13:20:37.421-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
GenericPipeline 2017-04-25T13:20:37.421-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
GenericPipeline 2017-04-25T13:20:37.422-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
GenericPipeline 2017-04-25T13:20:37.422-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:37.422-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:20:37.422-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:20:37.422-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:20:37.422-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:20:37.422-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:20:37.422-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:37.422-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:20:37.422-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:20:37.422-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:20:37.422-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:20:37.422-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:37.422511896 -0500 CDT
CapiNozzle 2017-04-25T13:20:37.423-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:20:37.423-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:20:37.424-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:20:37.424-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:37.424-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:20:37.426-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:20:37.427-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:20:37.431-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:37.426450665 -0500 CDT
DcpNozzle 2017-04-25T13:20:37.431-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:37.431828749 -0500 CDT
DcpNozzle 2017-04-25T13:20:37.434-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:20:37.434-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:20:37.434-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:37.434-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:20:37.434-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:37.434-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:20:37 Stream Closed for vb 171 StreamEnd simulated
AsyncListener 2017-04-25T13:20:37.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:20:37.434-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-489679894
AsyncListener 2017-04-25T13:20:37.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:37.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:37.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:37.434-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:20:37.434-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:37.434-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:20:37.435-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:20:37.435-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:20:37.437-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-489679894 is stopped
PipelineManager 2017-04-25T13:20:37.437-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:20:37.437-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:27.388285422-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 106 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:20:37.438-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:20:37.438-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:37.438-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:37.438-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:37.438-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.438-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:37.438-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:20:37.43503437 -0500 CDT
CapiNozzle 2017-04-25T13:20:37.439-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:20:37.439-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:20:37.439-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:20:37.440-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:20:37.440-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:37.440-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:20:37.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
XDCRFactory 2017-04-25T13:20:37.532-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:20:37.532-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:20:37.533-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:20:37.582-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:20:37.582-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:20:37.582-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:20:37.582-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:20:37.582-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:20:37.582-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:20:37.582-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:20:37.583-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:20:37.583-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
GenericSupervisor 2017-04-25T13:20:37.584-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:20:37.584-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
GenericSupervisor 2017-04-25T13:20:37.584-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:37.584-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:37.584-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:37.584-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:20:37.584-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:20:37.584-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
CheckpointManager 2017-04-25T13:20:37.584-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
CapiUtils 2017-04-25T13:20:37.603-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:20:37.603-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:20:37.603-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:20:37.603-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
StatisticsManager 2017-04-25T13:20:37.604-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:20:37.604-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
XDCRFactory 2017-04-25T13:20:37.604-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:20:37.604-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505 is constructed, start it
GenericPipeline 2017-04-25T13:20:37.605-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[223 259 281 188 189 248 197 288 326 200 304 313 190 199 209 216 236 256 287 269 253 261 242 271 303 305 327 254 249 174 231 273 307 336 298 340 196 306 310 173 250 290 312 202 218 243 181 339 297 229 316 226 251 275 337 224 212 247 317 318 333 334 177 206 238 262 274 222 233 270 277 301 213 198 237 268 279 195 332 330 203 239 246 325 185 266 175 180 278 322 338 258 292 314 324 186 219 257 315 320 235 204 272 176 194 214 210 225 267 295 241 264 331 280 302 255 191 311 321 328 329 178 183 187 193 211 244 335 172 245 282 289 182 220 252 230 293 296 179 208 284 184 228 240 319 234 286 309 217 207 283 291 299 323 285 300 265 294 232 221 227 192 308 201 205 215 260 276 171 263] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[121 6 60 66 82 86 90 117 59 52 3 150 105 156 37 87 96 131 28 0 155 107 71 68 51 129 26 48 170 157 12 99 135 1 67 125 49 72 75 126 34 81 102 143 145 31 84 130 40 133 62 127 137 24 168 114 20 85 124 41 43 8 70 7 110 61 119 160 65 148 140 80 46 141 33 22 167 57 111 142 152 166 50 118 14 18 69 104 122 162 4 103 113 132 23 56 83 144 158 163 17 106 116 54 109 161 88 89 146 63 93 115 35 154 95 128 138 165 169 16 64 101 108 25 55 97 112 39 92 94 42 15 38 73 153 11 147 29 123 9 47 78 98 30 45 77 36 74 79 134 149 44 139 27 10 19 91 151 159 5 120 53 100 21 13 76 136 32 164 58 2]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[224 212 247 317 318 333 334 177 206 238 262 274 222 233 270 277 301 213 198 237 268 279 195 332 330 203 239 246 325 185 266 175 180 278 322 338 258 292 314 324 186 219 257 315 320 235 204 272 176 194 214 210 225 267 295 241 264 331 280 302 255 191 311 321 328 329 178 183 187 193 211 244 335 172 245 282 289 182 220 252 230 293 296 179 208 284 184 228 240 319 234 286 309 217 207 283 291 299 323 285 300 265 294 232 221 227 192 308 201 205 215 260 276 171 263 223 259 281 188 189 248 197 288 326 200 304 313 190 199 209 216 236 256 287 269 253 261 242 271 303 305 327 254 249 174 231 273 307 336 298 340 196 306 310 173 250 290 312 202 218 243 181 339 297 229 316 226 251 275 337] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[85 124 41 43 8 70 7 110 61 119 160 65 148 140 80 46 141 33 22 167 57 111 142 152 166 50 118 14 18 69 104 122 162 4 103 113 132 23 56 83 144 158 163 17 106 116 54 109 161 88 89 146 63 93 115 35 154 95 128 138 165 169 16 64 101 108 25 55 97 112 39 92 94 42 15 38 73 153 11 147 29 123 9 47 78 98 30 45 77 36 74 79 134 149 44 139 27 10 19 91 151 159 5 120 53 100 21 13 76 136 32 164 58 2 121 6 60 66 82 86 90 117 59 52 3 150 105 156 37 87 96 131 28 0 155 107 71 68 51 129 26 48 170 157 12 99 135 1 67 125 49 72 75 126 34 81 102 143 145 31 84 130 40 133 62 127 137 24 168 114 20]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 log_level:Info replication_type:capi active:true worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:20:37.605-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505...
AsyncListener 2017-04-25T13:20:37.612-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:37.612-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:20:37.613-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:20:37.613-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:20:37.613-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:20:37.613-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:20:37.613-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:20:37.613-05:00 [INFO] Checkpointing starts in 1801.317 sec
PipelineRuntimeCtx 2017-04-25T13:20:37.613-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:20:37.613-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:20:37.613-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:20:37.613-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:20:37.614-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:20:37.614-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:20:37.614-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:20:37.614-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:20:37.615-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:20:37.641-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:20:37.641-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:20:37.641-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
StatisticsManager 2017-04-25T13:20:37.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:20:37.656-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:20:37.656-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
ToplogyChangeDetector 2017-04-25T13:20:37.659-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:20:37.669-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:20:37.669-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:20:37.669-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:20:37.683-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:20:37.705-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:20:37.705-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:20:37.705-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:20:37.705-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:20:37.705-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:20:37.705-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:20:37.724-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:20:37.724-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:20:37.724-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:20:37.748-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:20:37.748-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:20:37.812-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:20:37.907-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:20:37.907-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:20:37.907-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:20:37.908-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:20:37.908-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:20:37.908-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:20:37.908-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505 is started----------
PipelineManager 2017-04-25T13:20:37.908-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:20:37.908-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:20:37.908-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:20:38.085-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:20:38.086-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:20:38.086-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:20:38.086-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:20:38 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 323 successful
StatisticsManager 2017-04-25T13:20:38.703-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:20:38.751-05:00 [INFO] Send batch count=343
2017/04/25 13:20:38 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 130 successful
CapiNozzle 2017-04-25T13:20:38.858-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:38.903-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:20:38 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 332 successful
CapiNozzle 2017-04-25T13:20:38.910-05:00 [INFO] Send batch count=11
2017/04/25 13:20:38 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:20:38 UPR_STREAMREQ for vb 39 successful
CapiNozzle 2017-04-25T13:20:38.939-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:20:38.996-05:00 [ERROR] Received malformed response from tcp connection vb=1. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f204/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:20:39.252-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err Received malformed response from tcp connection vb=1. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f204/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:20:39.327-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:20:39 UPR_STREAMREQ for vb 241 successful
CapiNozzle 2017-04-25T13:20:39.779-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:20:40.241-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:20:40.277-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:20:40.263-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:20:43.519-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:20:43 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 339 successful
CapiNozzle 2017-04-25T13:20:43.520-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:20:43.536-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:20:43.537-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
2017/04/25 13:20:43 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 143 successful
StatisticsManager 2017-04-25T13:20:43.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
StatisticsManager 2017-04-25T13:20:43.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:20:43 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:20:43 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 239 successful
CheckpointManager 2017-04-25T13:20:44.007-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
CheckpointManager 2017-04-25T13:20:44.007-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:20:44 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 46 successful
CapiNozzle 2017-04-25T13:20:44.070-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:20:44.070-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:20:44.070-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:20:44.070-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:20:44.070-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
2017/04/25 13:20:44 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:20:44 UPR_STREAMREQ for vb 94 successful
DcpNozzle 2017-04-25T13:20:44.114-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:20:44.114-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
PipelineManager 2017-04-25T13:20:44.138-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:20:44.138-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:20:44.144-05:00 [INFO] err_list=[{"time":"2017-04-25T13:20:44.144935663-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:20:44.145-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:20:44.222-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:44.222-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:44.233-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:20:44.439-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:20:44.440-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:20:44.440-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:20:44.440-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:20:44.440-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:20:44.529-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:20:44.529-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:20:44.543-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
StatisticsManager 2017-04-25T13:20:44.627-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:20:44.629-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505 {"Errors": "[{\"time\":\"2017-04-25T13:20:44.144935663-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 166, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 2528, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 166, "set_filtered": 0, "set_received_from_dcp": 2528, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:20:45.045-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:20:45.056-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:20:45.241-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:20:45.282-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
GenericSupervisor 2017-04-25T13:20:50.591-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:20:50.714-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
ToplogyChangeDetector 2017-04-25T13:20:49.340-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:20:50.711-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:20:51.202-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
GenericSupervisor 2017-04-25T13:20:50.866-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:20:51.240-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
GenericSupervisor 2017-04-25T13:20:51.410-05:00 [INFO] Finish executing 0x681fb0
PipelineManager 2017-04-25T13:20:54.179-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
GenericSupervisor 2017-04-25T13:21:03.020-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
PipelineManager 2017-04-25T13:20:57.420-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:05.920-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
PipelineManager 2017-04-25T13:21:05.665-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:21:06.960-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
GenericSupervisor 2017-04-25T13:21:07.269-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineRuntimeCtx 2017-04-25T13:21:09.395-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:21:10.726-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:11.078-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:21:13.341-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:13.386-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:21:13.476-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:21:13.34740111 -0500 CDT
GenericSupervisor 2017-04-25T13:21:13.598-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:13.615-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:21:13.615-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:21:14.551-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:21:14.651-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:21:14.564-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:21:14.673-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
GenericSupervisor 2017-04-25T13:21:14.631-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:21:14.739-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:21:14.739-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:21:14.743-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:21:14.743-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-582810505 has been stopped
GenericPipeline 2017-04-25T13:21:15.765-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
GenericPipeline 2017-04-25T13:21:16.241-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
AsyncListener 2017-04-25T13:21:15.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:21:16.389-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:21:16.029-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:21:16.477-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:21:16.179-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:21:16.198-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:21:16.501-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.505-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.210-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:21:16.531-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:21:16.003-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
AsyncListener 2017-04-25T13:21:16.533-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:21:16.534-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericSupervisor 2017-04-25T13:21:15.636-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
AsyncListener 2017-04-25T13:21:16.539-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.539-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:21:16.506-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-582810505
GenericPipeline 2017-04-25T13:21:16.539-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.533-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:21:16.550-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:21:16.550-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:21:16.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:21:16.014-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:21:16.561-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.561-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:21:16.561-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:21:16.563-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:21:16.592-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.592-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.598-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:21:16.598-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:21:16.612-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:21:16.613-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:21:16.642-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:21:16.642-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:21:16.657-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
AsyncListener 2017-04-25T13:21:16.655-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.666-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:21:16.666-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.666-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:21:16.671-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:21:16.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:21:16.541-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:21:16.713-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:21:16.569-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:21:16.613-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:21:16.727-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:21:16.614-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
CapiNozzle 2017-04-25T13:21:16.761-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:21:16.714314382 -0500 CDT
DcpNozzle 2017-04-25T13:21:16.727-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:21:16.897-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericPipeline 2017-04-25T13:21:17.669-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:21:17.671-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:21:17.676-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:21:17.679-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:21:17.680-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:21:17.680-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
CapiNozzle 2017-04-25T13:21:16.891-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:21:16.72792874 -0500 CDT
CapiNozzle 2017-04-25T13:21:17.621-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:21:17.722-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:21:17.726-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:21:17.640-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:21:17.730-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:21:17.730-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:21:17.730-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
CapiNozzle 2017-04-25T13:21:17.731-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:21:17.734-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:21:17.736-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:21:17.736-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:21:17.736-05:00 [INFO] releasing capi client
DcpNozzle 2017-04-25T13:21:17.835-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:21:17.835-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:21:17.911-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-582810505 is stopped
PipelineManager 2017-04-25T13:21:17.911-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
DcpNozzle 2017-04-25T13:21:17.947-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:21:17.899990072 -0500 CDT
DcpNozzle 2017-04-25T13:21:17.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:21:17.975-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:21:17.950351961 -0500 CDT
DcpNozzle 2017-04-25T13:21:17.977-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
PipelineManager 2017-04-25T13:21:17.966-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:44.144935663-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:21:18.008-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:21:18.222-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:44.144935663-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:21:18.254-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:21:18.271-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:21:18.272-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
PipelineManager 2017-04-25T13:21:18.290-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:44.144935663-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={2 source nozzles have been constructed}
]
XDCRFactory 2017-04-25T13:21:18.336-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:21:18.351-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:21:18.456-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:21:18.471-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:21:18.473-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:21:18.473-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:21:18.482-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:21:18.526-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:21:18.540-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
GenericSupervisor 2017-04-25T13:21:18.556-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:21:18.559-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
GenericSupervisor 2017-04-25T13:21:18.561-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:18.561-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:18.561-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:18.561-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:18.561-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:21:18.569-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
CheckpointManager 2017-04-25T13:21:18.569-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
CapiUtils 2017-04-25T13:21:18.895-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:21:18.898-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:21:18.904-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:21:18.909-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
StatisticsManager 2017-04-25T13:21:18.933-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:21:18.935-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
XDCRFactory 2017-04-25T13:21:18.936-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:21:18.940-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 is constructed, start it
GenericPipeline 2017-04-25T13:21:18.952-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[300 171 196 207 247 282 240 244 175 245 297 327 271 225 295 290 306 185 235 241 173 303 191 193 223 325 309 335 178 264 318 273 308 263 184 233 324 224 229 299 337 321 305 226 192 183 289 210 269 236 274 284 179 278 204 217 304 328 174 277 298 246 261 288 176 221 253 267 270 292 203 234 311 252 280 315 172 232 205 222 248 177 276 283 334 254 301 200 333 340 268 242 293 320 250 302 186 209 262 251 231 237 307 219 265 312 314 329 182 230 279 216 291 187 201 275 323 326 190 258 259 266 338 310 206 227 331 180 272 322 255 281 339 197 220 199 285 286 330 195 194 294 332 313 317 238 243 296 319 198 287 316 208 249 256 257 260 336 218 239 202 211 212 213 188 181 189 214 215 228] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[49 135 76 112 138 141 51 70 56 142 20 111 2 128 156 96 114 140 12 17 22 55 60 139 149 72 18 26 36 53 44 54 62 90 11 27 59 117 146 163 0 10 73 85 83 107 133 31 120 124 42 13 144 9 87 91 132 134 14 75 95 126 99 21 145 152 160 63 79 131 170 4 105 24 39 147 1 25 3 52 94 108 159 64 77 80 93 61 164 167 5 69 157 81 86 123 23 106 115 47 113 121 122 154 57 116 66 102 109 67 78 127 136 74 19 100 71 168 43 45 153 28 41 118 151 104 110 129 165 68 82 161 6 29 65 162 30 35 38 103 137 89 143 58 88 148 84 92 130 155 8 150 32 33 15 50 101 119 158 7 16 97 98 34 46 169 166 37 40 48 125]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[136 67 78 127 74 19 100 71 168 43 45 153 151 28 41 118 165 104 110 129 68 82 161 6 29 65 103 162 30 35 38 137 89 143 58 88 148 155 84 92 130 8 150 32 33 15 50 98 101 119 158 7 16 97 34 46 169 125 166 37 40 48 49 135 141 76 112 138 51 70 56 142 20 111 2 128 156 96 55 114 140 12 17 22 60 139 149 72 18 26 36 53 44 54 62 90 11 27 59 85 117 146 163 0 10 73 83 107 133 31 120 124 42 13 132 144 9 87 91 126 134 14 75 95 99 160 21 145 152 170 63 79 131 4 105 24 39 147 1 25 3 52 94 93 108 159 64 77 80 61 164 167 5 69 157 81 86 123 23 106 115 47 113 121 122 154 57 116 66 102 109] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[265 312 314 329 279 182 230 201 216 291 187 323 275 190 326 258 259 266 310 338 206 227 272 331 180 322 220 255 281 339 197 195 199 285 286 330 194 294 332 243 313 317 238 287 296 319 198 257 316 208 249 256 239 260 336 218 202 211 212 213 188 181 189 214 215 228 247 300 171 196 207 244 282 240 175 245 297 327 271 295 225 306 290 173 185 235 241 223 303 191 193 309 325 318 335 178 264 308 273 233 263 184 299 324 224 229 337 305 321 192 226 183 289 269 210 284 236 274 179 278 204 217 304 328 298 174 277 246 261 221 288 176 234 253 267 270 292 203 311 315 252 280 232 172 177 205 222 248 276 283 334 200 254 301 268 333 340 242 293 320 250 302 209 186 262 237 251 231 219 307]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 active:true filter_expression: worker_batch_size:500 failure_restart_interval:10 target_nozzle_per_node:2 log_level:Info replication_type:capi]
CheckpointManager 2017-04-25T13:21:18.957-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329...
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:21:18.962-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:21:18.963-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:21:18.963-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:21:18.966-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:21:18.969-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:21:18.969-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:21:18.972-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:21:18.970-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:21:18.984-05:00 [INFO] Checkpointing starts in 1801.679 sec
PipelineRuntimeCtx 2017-04-25T13:21:18.986-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:21:18.986-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:21:18.994-05:00 [INFO] checkpointing rountine started
ToplogyChangeDetector 2017-04-25T13:21:18.995-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:21:19.011-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:21:19.011-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:21:19.013-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:21:19.019-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:21:19.021-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:21:19.027-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:21:19.047-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:21:19.091-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:21:19.130-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:21:19.144-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:21:19.149-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:21:19.151-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:21:19.252-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:21:19.369-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:21:19.699-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:21:20.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:21:20.692-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
DcpNozzle 2017-04-25T13:21:20.605-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:21:20.822-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
CapiNozzle 2017-04-25T13:21:20.925-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:21:21.488-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CapiNozzle 2017-04-25T13:21:21.457-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:21:21.515-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:21:21.529-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:21:21.512-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:21:21.513-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:21:21.465-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
DcpNozzle 2017-04-25T13:21:21.656-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:21:21.659-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:21:21.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:21:21.842-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:21:22.080-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:21:22.081-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:21:22.084-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
PipelineManager 2017-04-25T13:21:22.110-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:20:44.144935663-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:21:22.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:21:22.158-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:21:22.160-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:21:22.162-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 is started----------
PipelineManager 2017-04-25T13:21:22.165-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:21:22.165-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:21:22.166-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:21:22.194-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:21:22 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:21:22 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 53 successful
StatisticsManager 2017-04-25T13:21:23.109-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:21:23 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 251 successful
CapiNozzle 2017-04-25T13:21:23.139-05:00 [INFO] Send batch count=1
2017/04/25 13:21:23 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 151 successful
CapiNozzle 2017-04-25T13:21:23.183-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:21:23.191-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:21:23 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 253 successful
CapiNozzle 2017-04-25T13:21:23.264-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:21:23 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 61 successful
CapiNozzle 2017-04-25T13:21:23.298-05:00 [INFO] Send batch count=500
2017/04/25 13:21:23 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 83 successful
CapiNozzle 2017-04-25T13:21:23.640-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:21:23.661-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:21:23.661-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
2017/04/25 13:21:23 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 284 successful
CapiNozzle 2017-04-25T13:21:23.688-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:21:23 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:21:23 UPR_STREAMREQ for vb 96 successful
CheckpointManager 2017-04-25T13:21:23.763-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
DcpNozzle 2017-04-25T13:21:23.904-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CheckpointManager 2017-04-25T13:21:24.837-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:21:25 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:21:24.998-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:21:25 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:21:25 UPR_STREAMREQ for vb 98 successful
CapiNozzle 2017-04-25T13:21:25.368-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:21:25.323-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:21:26.245-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
GenericSupervisor 2017-04-25T13:21:29.154-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:21:29.870-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:21:36.323-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:21:37.704-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:21:38.567-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:21:40.241-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:21:40.294-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:40.297-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:21:40.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
GenericSupervisor 2017-04-25T13:21:40.223-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:40.386-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:21:40.488-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:41.856-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:21:41.843-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:43.107-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:21:43.930-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:21:51.990-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:21:51.903-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:21:52.196-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:21:52.295-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:21:52.425-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:21:52.546-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:21:53.865-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:21:53.625-05:00 [INFO] Executing 0x681fb0 timed out
StatisticsManager 2017-04-25T13:21:53.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:21:53.746-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
GenericSupervisor 2017-04-25T13:21:54.049-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:21:54.172-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:21:56.733-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:20:44.144935663-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 608, "dcp_datach_length": 413, "dcp_dispatch_time": 28, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 631, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 3, "docs_received_from_dcp": 29426, "docs_rep_queue": 24799, "docs_written": 2, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 8279, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 2, "set_failed_cr_source": 631, "set_filtered": 0, "set_received_from_dcp": 29426, "size_rep_queue": 32620326, "time_committing": 0, "wtavg_docs_latency": 886, "wtavg_meta_latency": 8016}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 356, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 316, "docs_opt_repd": 0, "docs_rep_queue": 20217, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 1, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 1, "set_failed_cr_source": 316, "size_rep_queue": 25300397, "wtavg_docs_latency": {"count": 1, "max": 162, "mean": 162, "min": 162}, "wtavg_meta_latency": {"count": 417, "max": 63, "mean": 61.721822541966425, "min": 61}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 252, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 315, "docs_opt_repd": 1, "docs_rep_queue": 15394, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 1, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 1, "set_failed_cr_source": 315, "size_rep_queue": 18541955, "wtavg_docs_latency": {"count": 1, "max": 1610, "mean": 1610, "min": 1610}, "wtavg_meta_latency": {"count": 422, "max": 15943, "mean": 15876.457345971565, "min": 14}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 16206, "max": 9999, "mean": 41.948, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 21677, "expiry_received_from_dcp": 0, "set_received_from_dcp": 21677}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 745, "dcp_dispatch_time": {"count": 13219, "max": 78, "mean": 11.03, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 17936, "expiry_received_from_dcp": 0, "set_received_from_dcp": 17936}}

CapiNozzle 2017-04-25T13:21:56.790-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:21:56.810-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
StatisticsManager 2017-04-25T13:21:56.883-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 22638 items, sent 501 items
StatisticsManager 2017-04-25T13:21:56.883-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 18567 items, sent 502 items
StatisticsManager 2017-04-25T13:21:56.981-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:22892]
StatisticsManager 2017-04-25T13:21:57.004-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 22892 items, sent 22892 items.
StatisticsManager 2017-04-25T13:21:57.004-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:19104]
StatisticsManager 2017-04-25T13:21:57.004-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 19104 items, sent 19104 items.
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:21:57.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:21:57.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:21:59.504-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
StatisticsManager 2017-04-25T13:21:59.521-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
GenericSupervisor 2017-04-25T13:22:00.880-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:22:01.909-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:22:02.218-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:22:02.356-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:22:02.358-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:22:02.360-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:22:02.644-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
ToplogyChangeDetector 2017-04-25T13:22:02.806-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:22:02.824-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
StatisticsManager 2017-04-25T13:22:02.853-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:20:44.144935663-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 608, "dcp_datach_length": 713, "dcp_dispatch_time": 12, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 631, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 3, "docs_received_from_dcp": 58941, "docs_rep_queue": 52971, "docs_written": 2, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 11540, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 2, "set_failed_cr_source": 631, "set_filtered": 0, "set_received_from_dcp": 58941, "size_rep_queue": 69171532, "time_committing": 0, "wtavg_docs_latency": 886, "wtavg_meta_latency": 8016}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 356, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 316, "docs_opt_repd": 0, "docs_rep_queue": 29740, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 1, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 1, "set_failed_cr_source": 316, "size_rep_queue": 38829437, "wtavg_docs_latency": {"count": 1, "max": 162, "mean": 162, "min": 162}, "wtavg_meta_latency": {"count": 417, "max": 63, "mean": 61.721822541966425, "min": 61}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 252, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 315, "docs_opt_repd": 1, "docs_rep_queue": 26770, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 1, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 1, "set_failed_cr_source": 315, "size_rep_queue": 35245946, "wtavg_docs_latency": {"count": 1, "max": 1610, "mean": 1610, "min": 1610}, "wtavg_meta_latency": {"count": 422, "max": 15943, "mean": 15876.457345971565, "min": 14}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 266, "dcp_dispatch_time": {"count": 31873, "max": 2561, "mean": 13.664, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 34721, "expiry_received_from_dcp": 0, "set_received_from_dcp": 34721}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 27068, "max": 405, "mean": 10.727, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 28561, "expiry_received_from_dcp": 0, "set_received_from_dcp": 28561}}

StatisticsManager 2017-04-25T13:22:02.873-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 28773 items, sent 502 items
StatisticsManager 2017-04-25T13:22:02.873-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 34938 items, sent 501 items
StatisticsManager 2017-04-25T13:22:02.875-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:34938]
StatisticsManager 2017-04-25T13:22:02.875-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 34938 items, sent 34938 items.
StatisticsManager 2017-04-25T13:22:02.875-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:28773]
StatisticsManager 2017-04-25T13:22:02.875-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 28773 items, sent 28773 items.
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:22:02.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:22:02.995-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
StatisticsManager 2017-04-25T13:22:03.070-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
PipelineManager 2017-04-25T13:22:03.106-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:22:03.441-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:22:03.587-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:22:03.637-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:22:03.638-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:22:03.638-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:22:03.656-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:22:03.698-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:22:03.781-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:22:04.102-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:22:04.265-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:22:04.064-05:00 [INFO] err_list=[{"time":"2017-04-25T13:22:03.883348132-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
CapiNozzle 2017-04-25T13:22:05.336-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:22:06.125-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:22:06.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
PipelineManager 2017-04-25T13:22:06.178-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
GenericSupervisor 2017-04-25T13:22:06.129-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:22:06.232-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:22:06.240-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 {"Errors": "[{\"time\":\"2017-04-25T13:22:03.883348132-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 608, "dcp_datach_length": 266, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 631, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 3, "docs_received_from_dcp": 66218, "docs_rep_queue": 0, "docs_written": 2, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 2, "set_failed_cr_source": 631, "set_filtered": 0, "set_received_from_dcp": 66218, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

ToplogyChangeDetector 2017-04-25T13:22:10.017-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:22:10.053-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:22:56.067-05:00 [ERROR] _revs_diff query for vb 300 failed with err=Post http://10.202.98.33:9091/metadata%2f300/_revs_diff: EOF
CapiNozzle 2017-04-25T13:22:56.074-05:00 [ERROR] batchGetMeta failed. err=Post http://10.202.98.33:9091/metadata%2f300/_revs_diff: EOF
GenericSupervisor 2017-04-25T13:22:55.984-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:22:56.216-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:22:56.229-05:00 [INFO] ****************************
PipelineManager 2017-04-25T13:22:56.296-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:22:56.297-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:22:56.298-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:22:56.299-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
PipelineRuntimeCtx 2017-04-25T13:22:56.301-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:22:56.305-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:22:56.306-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:22:56.307-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
CheckpointManager 2017-04-25T13:22:56.320-05:00 [INFO] Pipeline is no longer running, exit.
CheckpointManager 2017-04-25T13:22:56.320-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
GenericSupervisor 2017-04-25T13:22:56.307-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:22:56.366-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:22:56.419-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:22:56.421-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:22:56.374952967 -0500 CDT
GenericSupervisor 2017-04-25T13:22:56.428-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:22:56.428-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:22:56.429-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:22:56.429-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:22:56.429-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:22:56.432-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:22:56.432-05:00 [INFO] Exits checkpointing routine.
PipelineRuntimeCtx 2017-04-25T13:22:56.432-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 has been stopped
GenericPipeline 2017-04-25T13:22:56.446-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-481596329 is stopped
PipelineManager 2017-04-25T13:22:56.446-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:22:56.450-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
GenericPipeline 2017-04-25T13:22:56.450-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
GenericPipeline 2017-04-25T13:22:56.450-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
GenericPipeline 2017-04-25T13:22:56.450-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-481596329
AsyncListener 2017-04-25T13:22:56.451-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:22:56.454-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:22:56.454-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:22:56.455-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:22:56.455-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:22:56.455-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:22:56.456-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:22:56.456-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:22:56.456-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:22:56.456-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:22:56.456-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:22:56.456-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:22:56.457-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:22:56.458-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:22:56.460-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:22:56.466-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:22:56.467-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:22:56.468-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:22:56.468-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:22:56.468-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:22:56.468-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:22:56.468-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:22:56.468-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:22:56.468-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:22:56.468-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:22:56.468301626 -0500 CDT
CapiNozzle 2017-04-25T13:22:56.468-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:22:56.468642063 -0500 CDT
CapiNozzle 2017-04-25T13:22:56.468-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:22:56.469-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:22:56.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
panic: send on closed channel

goroutine 3196 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc3b4f9dd40, 0x8edb60, 0xc2bab6c230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc3b5201f80, 0x8ed920, 0xc3e1facbb0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3b4f76600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 4 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 4 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc20802b960)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080ea001, 0x5ff, 0x5ff, 0xc20800bc80, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080ea001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080e4000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e4000, 0x8f07a0, 0xc2080e6020, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc2080e6000, 0xc20801fbe0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc2080de000, 0x7f782990bc10, 0xc2080e6000, 0x0, 0x0, 0xc20801fbe0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc2080de000, 0x7f782990bc10, 0xc2080e6000, 0xc2080e0000, 0xc2080dc060, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc2080de000, 0x7f782990bc10, 0xc2080e6000)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080c0480, 0xc2080bbf90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080bbf90, 0xc2080c0480, 0x7f782990a398, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080c0480, 0xc2080c04b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 16 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc20807c540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 12 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc208078c60)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 13 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc208078c60)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 15 [syscall, 4 minutes]:
syscall.Syscall(0x0, 0x0, 0xc2080f5000, 0x1000, 0xc20803b1d0, 0xc2080326e0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc2080f5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc2080f5000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc2080f5000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc2080f5000, 0x1000, 0x1000, 0xc2080d9630, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc2080e69c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc2080e69c0, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 3127 [runnable]:
encoding/binary.bigEndian.Uint64(0xc3e1e66820, 0x8, 0x19e, 0x15b)
	/usr/local/go/src/encoding/binary/binary.go:109
github.com/couchbase/gomemcached/client.makeUprEvent(0x57, 0x14b31f0ea7660000, 0x25bfbf0025, 0xc3e1e66820, 0x1f, 0x19e, 0xc3e1e6683f, 0x24, 0x17f, 0xc3e1e66863, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:111 +0x158
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3b5337cb0, 0xc3b9d048a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:476 +0x1cc5
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 33 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc20807c550)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 34 [semacquire]:
sync.(*WaitGroup).Wait(0xc2080f00c0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:191 +0x428
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 35 [select, 4 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 36 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc208092040, 0x11, 0xc208062fc0, 0xc2080e65a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc208092040, 0x11, 0xc208062fc0, 0xc2080e65a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc208092040, 0x11, 0xc208062fc0, 0xc2080e65a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080d8ff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 37 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080d5b10, 0xf, 0xc208064fc0, 0xc2080e65a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080d5b10, 0xf, 0xc208064fc0, 0xc2080e65a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080d5b10, 0xf, 0xc208064fc0, 0xc2080e65a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080d9180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 38 [select]:
net/http.(*persistConn).roundTrip(0xc3c2d5e0b0, 0xc2416e8ce0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc3e13b96c0, 0xc2ba83f900, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3e13b96c0, 0x7f7829909fe8, 0xe94600, 0x49, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3e13b96c0, 0x49, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3e13b96c0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3e13b96c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3e13b96c0, 0x7f782990db40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3e0a3d0a0, 0xc35465cb80, 0x34, 0x7f782990db40, 0xc3e09beb40, 0x958760, 0xc3030c27e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc3e0a41a40, 0xc35465cb80, 0x34, 0x958760, 0xc3030c27e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Client).GetPool(0xc3e0a41a40, 0x9cae60, 0x7, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:767 +0x339
github.com/couchbase/goxdcr/utils.LocalPool(0xc3e0a0a9c0, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:89 +0x4a7
github.com/couchbase/goxdcr/utils.LocalBucketUUID(0xc3e0a0a9c0, 0xe, 0xc2080d5768, 0x8, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:370 +0x73
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080fa150, 0xc3e081b680, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:464 +0x12d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080fa150, 0xc3e081b680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc2080fa930)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 39 [chan send]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc2080e6960)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:110 +0x737
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 24 [select]:
net/http.(*persistConn).roundTrip(0xc208094160, 0xc24cf46060, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc3502dfba0, 0xc2b613c960, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3502dfba0, 0x7f7829909fe8, 0xc20802c090, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc3502dfba0, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc3502dfba0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc3502dfba0, 0xeb0d80, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/goxdcr/utils.doRestCall(0xc3502dfba0, 0x0, 0x818220, 0xc3d979e0d0, 0xeb0d80, 0xc20802b450, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc217284240, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:173 +0x201
github.com/couchbase/goxdcr/utils.QueryRestApi(0xc217284240, 0x15, 0x9e0430, 0xc, 0xc20c617600, 0x9bf4c0, 0x3, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:134 +0x17a
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostInfo(0xc20801f440, 0xc269bbe900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:113 +0x139
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:157 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostName(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:177 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyHost(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:50 +0x4c
github.com/couchbase/goxdcr/replication_manager.GetReplicationInfos(0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:507 +0x304
github.com/couchbase/goxdcr/replication_manager.(*Adminport).doGetAllReplicationInfosRequest(0xc2080e6960, 0xc3e1cdb2b0, 0xc269bbe690, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:361 +0x73
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc2080e6960, 0xc3e1cdb2b0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:188 +0x7d9
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc2080e6960, 0xc3547c6270, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc3547c6270, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080e6978)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 22 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc208094000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 23 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc208094000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 25 [IO wait]:
net.(*pollDesc).Wait(0xc208010920, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010920, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080108c0, 0x0, 0x7f7829909df0, 0xc2914f30e0)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038128, 0xc20805fe38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208038128, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208066ea0, 0x7f782990d760, 0xc208038128, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 26 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc2080115d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080115d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011570, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc2080d5c68)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038178, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc208038178, 0xc208078cb8, 0xc2080d3000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801fec0, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080667e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080667e0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080667e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080c10b0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080c10b0, 0xc208129200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc208102600, 0xc208129200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc208102600, 0xc208129200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc208102640, 0xc208129200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080e4300, 0xc20800bd00, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e4300, 0x80f160, 0xc208102700, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 27 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc208010760, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010760, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010700, 0xc208156000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc20802b9e8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380d0, 0xc208156000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc2080380d0, 0xc208094058, 0xc208156000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080f0240, 0xc208156000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208066a80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208066a80, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208066a80, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080c11a0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080c11a0, 0xc2080ea600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc208102780, 0xc2080ea600, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc208102780, 0xc2080ea600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2081027c0, 0xc2080ea600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080e4600, 0xc208102100, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e4600, 0x80f160, 0xc208102880, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 70 [select]:
net/http.(*persistConn).writeLoop(0xc208094160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 69 [runnable]:
net.(*pollDesc).Wait(0xc2080107d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080107d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010770, 0xc208175000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3df7c0078)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380d8, 0xc208175000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc2080380d8, 0xc2080941b8, 0xc208175000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081d6060, 0xc208175000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e6600)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e6600, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208094160)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 394 [select]:
net/http.(*persistConn).writeLoop(0xc2416c2210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 317 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x65, 0xc22ccb2000, 0xec, 0x100, 0xc22c4a9800, 0xc6, 0x100, 0xc22cca0420, 0xc22cca0360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3080 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3b52de0f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 55 [IO wait]:
net.(*pollDesc).Wait(0xc2080faf40, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080faf40, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080faee0, 0xc208152000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3d9b95808)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f8290, 0xc208152000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc2080f8290, 0xc208094738, 0xc208152000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208101fe0, 0xc208152000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e7380)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e7380, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080946e0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 56 [select]:
net/http.(*persistConn).writeLoop(0xc2080946e0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1837 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b680, 0xc9, 0xc3358d0500, 0xec, 0x100, 0xc33cdf5000, 0xbd, 0x100, 0xc33d66eae0, 0xc33d66ea20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2545 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1200, 0x65, 0xc3673e5600, 0xec, 0x100, 0xc369b92000, 0xc6, 0x100, 0xc369c5ce40, 0xc369c5cd80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 515 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2412c1d40, 0x64, 0xc220a0fa00, 0xeb, 0x100, 0xc23fb83200, 0x2c, 0x40, 0xc23a515d40, 0xc23a515c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3137 [runnable]:
github.com/couchbase/goxdcr/parts.getSerializedRevision(0xc3c063b740, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:1007 +0x355
github.com/couchbase/goxdcr/parts.getDocMap(0xc3c063b740, 0xc2b6386000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:980 +0x1c7
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3b4f9dd40, 0x40012c, 0xc3c38d7b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3b4f9dd40, 0xc3c079012c, 0xc3c38d7b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3b4f9dd40, 0xc3be18e690, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3b4f9dd40, 0xc3be18e690, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3b4f9dd40, 0xc3b51b4230, 0xc3b4f9dee0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 875 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2d80, 0x65, 0xc27bfe8d00, 0xec, 0x100, 0xc286205000, 0xc6, 0x100, 0xc286dccf00, 0xc286dcce40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1535 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93eab40, 0xca, 0xc2ef8c8400, 0xec, 0x100, 0xc2eeee6800, 0xbc, 0x100, 0xc2efb82a80, 0xc2efb829c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2233 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75440, 0x64, 0xc30c301700, 0xec, 0x100, 0xc339eb0000, 0xc0, 0x100, 0xc2ea2b6660, 0xc2ea2b65a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1204 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6240, 0x1, 0xc2b62d5100, 0xea, 0x100, 0xc2b60db800, 0xc7, 0x100, 0xc2b6a0fec0, 0xc2b6a0fe00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3608 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xc20807c540, 0xc2b613ca80, 0xc2b613cb10, 0xc2080fa700, 0xc2b444e230, 0xc2080f00a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:269 +0xa90
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 2629 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1440, 0x12c, 0xc3890abc00, 0xec, 0x100, 0xc38ec48800, 0xc4, 0x100, 0xc38fb25bc0, 0xc38fb25b00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1400 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93eab40, 0x12c, 0xc27bfe9700, 0xec, 0x100, 0xc2d516ec00, 0x21, 0x40, 0xc2d640f560, 0xc2d640f4a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1876 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b440, 0x4, 0xc3358d1d00, 0xea, 0x100, 0xc34e389000, 0xa2, 0x100, 0xc34e671980, 0xc34e6718c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3634 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc235e84160, 0x1dcd6500, 0xc3b51c5d20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3b52de0f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f7829913d28, 0xc3b52de0f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2593 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1200, 0x6a, 0xc3787b2000, 0xec, 0x100, 0xc377e78800, 0xe8, 0x100, 0xc378719a40, 0xc378719980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1502 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93ea900, 0x64, 0xc23f650800, 0xec, 0x100, 0xc2d9417800, 0xc9, 0x100, 0xc2ad0d58c0, 0xc2ad0d5800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2303 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75680, 0xcb, 0xc2eb2d6d00, 0xec, 0x100, 0xc314a6e800, 0xc9, 0x100, 0xc335b03140, 0xc335b03080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 302 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x0, 0xc220a0e900, 0xea, 0x100, 0xc220454800, 0xb8, 0x100, 0xc220c5ea80, 0xc220c5e9c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3519 [chan receive]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc3d8ec91e0, 0x9cac80, 0x3, 0xc2080d5768)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc3b5172700, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc3b5172700, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3d43f30e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 393 [IO wait]:
net.(*pollDesc).Wait(0xc2416de990, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2416de990, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2416de930, 0xc225bec000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3c61b8ef8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2417aa098, 0xc225bec000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc2417aa098, 0xc2416c2268, 0xc225bec000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc22acc8260, 0xc225bec000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc22359dbc0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc22359dbc0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2416c2210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1091 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6240, 0x65, 0xc208055a00, 0xeb, 0x100, 0xc2a4a7f200, 0x20, 0x20, 0xc2a4a6b020, 0xc2a4a6af60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3622 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21d3d0f60, 0x3b9aca00, 0xc3b51c5a80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3b4f91340, 0x7f7829913ca8, 0xc3b4f76300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f7829913ca8, 0xc3b4f76300)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3430 [select]:
net/http.(*persistConn).writeLoop(0xc3c49dd8c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2653 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1440, 0xcc, 0xc3965d0800, 0xec, 0x100, 0xc395e09000, 0xb7, 0x100, 0xc39674d9e0, 0xc39674d920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1880 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b680, 0x12e, 0xc34ed82000, 0xec, 0x100, 0xc34ea0f000, 0xb3, 0x100, 0xc34ed72540, 0xc34ed72480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2615 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1440, 0x12d, 0xc37eb36700, 0xec, 0x100, 0xc37e5fb000, 0xcf, 0x100, 0xc37edaf8c0, 0xc37edaf800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3427 [IO wait]:
net.(*pollDesc).Wait(0xc3c4a7e450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c4a7e450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c4a7e3f0, 0xc3c5085000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3c4fbce50)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3c41a3540, 0xc3c5085000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc3c41a3540, 0xc3c49dd9c8, 0xc3c5085000, 0x1000, 0x1000, 0x7f78298f84c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3c4fb83a0, 0xc3c5085000, 0x1000, 0x1000, 0xc1ec3e5ca9, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c4f05ec0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c4f05ec0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c49dd970)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 130 [select]:
net/http.(*persistConn).writeLoop(0xc208078dc0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 129 [IO wait]:
net.(*pollDesc).Wait(0xc20c612290, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c612290, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c612230, 0xc20c61e000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3d95f0990)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038228, 0xc20c61e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc208038228, 0xc208078e18, 0xc20c61e000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c60ec40, 0xc20c61e000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20c61c360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20c61c360, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208078dc0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1515 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93ea900, 0x66, 0xc226001100, 0xec, 0x100, 0xc292ae0000, 0xc8, 0x100, 0xc2d0d12ae0, 0xc2d0d12a20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 665 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc241524000, 0x12f, 0xc208054e00, 0xec, 0x100, 0xc26aaab800, 0xce, 0x100, 0xc26aec6e40, 0xc26aec6d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 943 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2d80, 0x66, 0xc22ccb2b00, 0xec, 0x100, 0xc29aeb7000, 0xc8, 0x100, 0xc29b256720, 0xc29b256660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1551 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93ea900, 0x0, 0xc2fdc00e00, 0xea, 0x100, 0xc2fd222800, 0xb4, 0x100, 0xc2fddc63c0, 0xc2fddc6300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 145 [syscall, 4 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 2324 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75440, 0x2, 0xc24e2d3200, 0xea, 0x100, 0xc2f2985800, 0xcf, 0x100, 0xc30c097800, 0xc30c097740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 884 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2fc0, 0x12c, 0xc27bedff00, 0xec, 0x100, 0xc289161800, 0x98, 0x100, 0xc289860480, 0xc2898603c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 335 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xca, 0xc220a0f400, 0xec, 0x100, 0xc22c60b000, 0xbc, 0x100, 0xc21cdaf7a0, 0xc21cdaf6e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 812 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2d80, 0x0, 0xc274e55700, 0xe9, 0x100, 0xc27574a900, 0x15, 0x20, 0xc2754e1140, 0xc2754e1080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 644 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2412c1d40, 0x0, 0xc2408c8e00, 0xea, 0x100, 0xc25624f800, 0xb8, 0x100, 0xc230f7bbc0, 0xc230f7bb00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3628 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2369ebfe0, 0x1dcd6500, 0xc3b51c5d40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3b52de460, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f7829913d28, 0xc3b52de460)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1850 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b440, 0x66, 0xc33adb8e00, 0xec, 0x100, 0xc33accf800, 0xc8, 0x100, 0xc33b2ccb40, 0xc33b2cca80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 294 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x12c, 0xc208168c00, 0xec, 0x100, 0xc219c95800, 0xd1, 0x100, 0xc21a089e60, 0xc21a089da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3640 [runnable]:
sync.(*RWMutex).Lock(0xc3be416240)
	/usr/local/go/src/sync/rwmutex.go:87 +0xa4
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3be416240, 0xc53300b9, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:309 +0x56
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3b4f76600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3b4f76600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccb380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3071 [semacquire]:
sync.(*WaitGroup).Wait(0xc3b4f9dca0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc3b4f9db00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3b4f9db00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 817 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2fc0, 0x12c, 0xc275938b00, 0xec, 0x100, 0xc275c80c00, 0x43, 0x80, 0xc275cf40c0, 0xc275cf4000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1240 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6240, 0x66, 0xc208168400, 0xec, 0x100, 0xc2c8440000, 0xc8, 0x100, 0xc22e838180, 0xc22e8380c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 599 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2412c1d40, 0x1, 0xc24e2d2400, 0xea, 0x100, 0xc24dcf8800, 0xc7, 0x100, 0xc24e326ae0, 0xc24e326a20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1484 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93eab40, 0xc8, 0xc23a55a500, 0xec, 0x100, 0xc2d6524000, 0xbb, 0x100, 0xc2a6029aa0, 0xc2a60299e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3193 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3b4f76600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3126 [runnable]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3b5561c20, 0xc3b9205a40, 0xc3b9205aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 365 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xcb, 0xc23a55a800, 0xec, 0x100, 0xc23a486000, 0xc9, 0x100, 0xc23a551320, 0xc23a551260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 327 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x130, 0xc220a0f700, 0xec, 0x100, 0xc223a7a000, 0xbc, 0x100, 0xc223a57e00, 0xc223a57d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 377 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x12d, 0xc2408c9000, 0xec, 0x100, 0xc2409e5000, 0xcf, 0x100, 0xc240e96240, 0xc240e96180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1096 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6480, 0xc8, 0xc2a505c600, 0xec, 0x100, 0xc2a52fa000, 0x34, 0x40, 0xc2a5378120, 0xc2a5378060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3429 [IO wait]:
net.(*pollDesc).Wait(0xc3c4a7e3e0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c4a7e3e0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c4a7e380, 0xc3c5087000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3c4fbced8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3c41a3548, 0xc3c5087000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc3c41a3548, 0xc3c49dd918, 0xc3c5087000, 0x1000, 0x1000, 0x7f78298f84c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3c4fb8400, 0xc3c5087000, 0x1000, 0x1000, 0xc1ec3e5ca8, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c4f05f20)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c4f05f20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c49dd8c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 669 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2412c1d40, 0x64, 0xc25d3fbc00, 0xec, 0x100, 0xc26b28f000, 0xb0, 0x100, 0xc26b681020, 0xc26b680f60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1805 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b440, 0x2, 0xc314362600, 0xea, 0x100, 0xc313e9d800, 0xcf, 0x100, 0xc3144a5e00, 0xc3144a5d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3648 [chan receive]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccb980)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3070 [runnable]:
bytes.(*Buffer).Write(0xc208078f20, 0xc268ef2038, 0x400, 0x400, 0x0, 0x0, 0x0)
	/usr/local/go/src/bytes/buffer.go:125
encoding/base64.(*encoder).Write(0xc268ef2000, 0xc3b9de5043, 0x443, 0x443, 0xc00, 0x0, 0x0)
	/usr/local/go/src/encoding/base64/base64.go:163 +0x449
encoding/json.encodeByteSlice(0xc208078f20, 0x8271c0, 0xc21d38f9e0, 0x57, 0xc21d38f900)
	/usr/local/go/src/encoding/json/encode.go:648 +0x3ac
encoding/json.(*encodeState).reflectValue(0xc208078f20, 0x8271c0, 0xc21d38f9e0, 0x57)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.interfaceEncoder(0xc208078f20, 0x833e80, 0xc240d569f0, 0x54, 0xc240d56900)
	/usr/local/go/src/encoding/json/encode.go:557 +0xd9
encoding/json.(*mapEncoder).encode(0xc2080f8040, 0xc208078f20, 0x8374c0, 0xc2ba831ec0, 0x15, 0x0)
	/usr/local/go/src/encoding/json/encode.go:619 +0x33d
encoding/json.*mapEncoder.(encoding/json.encode)·fm(0xc208078f20, 0x8374c0, 0xc2ba831ec0, 0x15, 0xc2ba831e00)
	/usr/local/go/src/encoding/json/encode.go:629 +0x58
encoding/json.(*encodeState).reflectValue(0xc208078f20, 0x8374c0, 0xc2ba831ec0, 0x15)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.(*encodeState).marshal(0xc208078f20, 0x8374c0, 0xc2ba831ec0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:270 +0xca
encoding/json.Marshal(0x8374c0, 0xc2ba831ec0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:135 +0x92
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3b4f9db00, 0x65, 0xc2080b5b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:785 +0x34b
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3b4f9db00, 0xc3bf3d0065, 0xc2080b5b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3b4f9db00, 0xc3b97705b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3b4f9db00, 0xc3b97705b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3b4f9db00, 0xc3b51b40e0, 0xc3b4f9dca0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 3618 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066e40, 0x7f782990d810, 0xc3e04aa6e0, 0xc3502de000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7f782990d810, 0xc3e04aa6e0, 0xc3502de000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f782990d810, 0xc3e04aa6e0, 0xc3502de000)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066ea0, 0x7f782990d810, 0xc3e04aa6e0, 0xc3502de000)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3e090c280)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3561 [runnable]:
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc2698b6000, 0x7f782990bb08, 0xc3d979e5e0, 0xc3da104560, 0x18, 0x18, 0x0, 0x7f782990bb08, 0xc2412fdbe0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:147
github.com/couchbase/gomemcached/client.getResponse(0x7f782990bb08, 0xc3d979e5e0, 0xc3da104560, 0x18, 0x18, 0xc2698b6000, 0x99, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc3da1009c0, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x25d
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc3da1009c0, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc3d8ec91e0, 0x2, 0x9cac80, 0x3, 0xc3d9993aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 357 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x1, 0xc22ccb2d00, 0xea, 0x100, 0xc230f96800, 0xc7, 0x100, 0xc2313f0240, 0xc2313f0180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 369 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x66, 0xc23f650e00, 0xec, 0x100, 0xc23fa5e000, 0xc8, 0x100, 0xc23fb84120, 0xc23fb84060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 267 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x64, 0xc208054300, 0xec, 0x100, 0xc219c97000, 0xa4, 0x100, 0xc21a21d4a0, 0xc21a21d3e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 634 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc241524000, 0xc8, 0xc24e2d2200, 0xec, 0x100, 0xc25a1b3000, 0xba, 0x100, 0xc234236c00, 0xc234236b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1751 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b680, 0xcc, 0xc30c301500, 0xec, 0x100, 0xc30cfe3800, 0x3a, 0x40, 0xc30d101740, 0xc30d101680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2315 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75680, 0xca, 0xc25d3fa400, 0xec, 0x100, 0xc2ff4b4000, 0xbc, 0x100, 0xc2ff4ff140, 0xc2ff4ff080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1519 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93eab40, 0x12c, 0xc215e61000, 0xec, 0x100, 0xc29b0fd000, 0xb2, 0x100, 0xc29269e060, 0xc2923a9f80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2498 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1200, 0x6a, 0xc220a0fd00, 0xec, 0x100, 0xc363f0d200, 0x36, 0x40, 0xc364448a20, 0xc364448960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1712 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b440, 0x3, 0xc30c418f00, 0xea, 0x100, 0xc30c4bc600, 0x35, 0x40, 0xc30c5036e0, 0xc30c503620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3632 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc235e840e0, 0x1dcd6500, 0xc3b51c5b30, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3b4fe7220, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f7829913d28, 0xc3b4fe7220)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3638 [chan send]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3b5337cb0, 0xc5330000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3b4f76300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3b4f76300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccb200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3641 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3b4f9dd40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3b4f9dd40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccb440)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2655 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1200, 0x69, 0xc3965d1f00, 0xec, 0x100, 0xc3970d0000, 0xba, 0x100, 0xc39713c3c0, 0xc39713c300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1214 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6480, 0x12c, 0xc2d27be300, 0xec, 0x100, 0xc2d2339800, 0xc4, 0x100, 0xc2d2902840, 0xc2d2902780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1266 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6240, 0x65, 0xc2bb618f00, 0xec, 0x100, 0xc2b6b14800, 0xad, 0x100, 0xc2df531200, 0xc2df531140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2265 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75680, 0xcd, 0xc30c418c00, 0xec, 0x100, 0xc33d92b800, 0xba, 0x100, 0xc2b33cb440, 0xc2b33cb380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2291 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75680, 0x12e, 0xc2f9433400, 0xec, 0x100, 0xc2a4ad0800, 0xb3, 0x100, 0xc32d05e9c0, 0xc32d05e900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 900 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2fc0, 0x12d, 0xc220a0f500, 0xec, 0x100, 0xc289164800, 0xcf, 0x100, 0xc28dba1f20, 0xc28dba1e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2097 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75440, 0x64, 0xc2a8589e00, 0xeb, 0x100, 0xc2b69fe600, 0x15, 0x20, 0xc2b89061e0, 0xc2b8906120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1232 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6480, 0x12e, 0xc2d27bf200, 0xec, 0x100, 0xc2aa3bb000, 0xb3, 0x100, 0xc2dbbeef00, 0xc2dbbeee40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1567 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93eab40, 0xc9, 0xc2fee19100, 0xec, 0x100, 0xc2fed49800, 0xbd, 0x100, 0xc2ff16b1a0, 0xc2ff16b0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1210 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6240, 0x0, 0xc2ca187400, 0xea, 0x100, 0xc2c9e90800, 0xb8, 0x100, 0xc2ca4bec00, 0xc2ca4beb40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 759 [select]:
net/http.(*persistConn).writeLoop(0xc26c135970)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 758 [IO wait]:
net.(*pollDesc).Wait(0xc26c76aca0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc26c76aca0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc26c76ac40, 0xc270a56000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3d85840e0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc26b87d8b8, 0xc270a56000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc26b87d8b8, 0xc26c1359c8, 0xc270a56000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc270a524e0, 0xc270a56000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc270a1c8a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc270a1c8a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc26c135970)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2605 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1200, 0x2, 0xc387230600, 0xea, 0x100, 0xc387372000, 0xcf, 0x100, 0xc38743ca80, 0xc38743c9c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1177 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29bac6480, 0xc9, 0xc2b9558000, 0xec, 0x100, 0xc2b8ac2800, 0xbd, 0x100, 0xc2b94e51a0, 0xc2b94e50e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 928 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2fc0, 0xc8, 0xc24e2d2700, 0xec, 0x100, 0xc29897e800, 0xbb, 0x100, 0xc298e539e0, 0xc298e53920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 941 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2fc0, 0xc9, 0xc23f651b00, 0xec, 0x100, 0xc29a7a9800, 0xbd, 0x100, 0xc29aadda40, 0xc29aadd980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3431 [IO wait]:
net.(*pollDesc).Wait(0xc3c4a7e370, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c4a7e370, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c4a7e310, 0xc3c5089000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3c4fbcf00)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3c41a3550, 0xc3c5089000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc3c41a3550, 0xc3c49dd868, 0xc3c5089000, 0x1000, 0x1000, 0x7f78298f84c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3c4fb8460, 0xc3c5089000, 0x1000, 0x1000, 0xc1ec3e5ca8, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c4f05f80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c4f05f80, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c49dd810)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 909 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2d80, 0x64, 0xc26bfc5f00, 0xec, 0x100, 0xc2994dc000, 0xc9, 0x100, 0xc29958c000, 0xc2994d7f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3371 [select]:
net/http.(*persistConn).writeLoop(0xc3c2d5e0b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 916 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26c2f2d80, 0x0, 0xc27bedf500, 0xea, 0x100, 0xc290747800, 0xab, 0x100, 0xc28ccece40, 0xc28ccecd80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3428 [select]:
net/http.(*persistConn).writeLoop(0xc3c49dd970)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3575 [runnable]:
github.com/couchbase/goxdcr/log.(*CommonLogger).Infof(0xc2080d59f0, 0xa130d0, 0x16, 0xc363a19d58, 0x1, 0x1)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:147
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).stopPipeline(0xeb0fa0, 0xc20802d560, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:350 +0x90c
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc3def2ef80, 0xc3e0b15c20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:583 +0x274
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc3def2ef80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 3074 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3b52de460)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 1531 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d93ea900, 0x1, 0xc2eb2d7f00, 0xea, 0x100, 0xc2ed181000, 0xc7, 0x100, 0xc2eea643c0, 0xc2eea64300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1808 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b680, 0xc8, 0xc314363b00, 0xec, 0x100, 0xc315e5e800, 0xbb, 0x100, 0xc316895a40, 0xc316895980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1818 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b440, 0x3, 0xc322601000, 0xea, 0x100, 0xc324792800, 0x9e, 0x100, 0xc324db6780, 0xc324db66c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1822 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc302a9b680, 0x12d, 0xc326e1aa00, 0xec, 0x100, 0xc3269fd000, 0xcf, 0x100, 0xc3275a0780, 0xc3275a06c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3370 [runnable]:
net.(*pollDesc).Wait(0xc3c2fa4a70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c2fa4a70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c2fa4a10, 0xc3c391e000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3e1e37b70)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3c30be660, 0xc3c391e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc3c30be660, 0xc3c2d5e108, 0xc3c391e000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3c37e38e0, 0xc3c391e000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c38ebf80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c38ebf80, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c2d5e0b0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3603 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066e40, 0x7f782990d810, 0xc3e090c000, 0xc3e1cdb1e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7f782990d810, 0xc3e090c000, 0xc3e1cdb1e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f782990d810, 0xc3e090c000, 0xc3e1cdb1e0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066ea0, 0x7f782990d810, 0xc3e090c000, 0xc3e1cdb1e0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3e090c0a0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2305 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75440, 0x66, 0xc2eb2d7c00, 0xec, 0x100, 0xc30fb8e800, 0xc8, 0x100, 0xc2f8371ec0, 0xc2f8371e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2633 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1200, 0x66, 0xc390fbeb00, 0xec, 0x100, 0xc3911e2000, 0xc8, 0x100, 0xc3911fdf20, 0xc3911fde60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3623 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21d3d1160, 0x3b9aca00, 0xc3b51c5a80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3b4f91340, 0x7f7829913900, 0xc3b4f9db00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f7829913900, 0xc3b4f9db00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3078 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3b4fe7220)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2950 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39fd47200, 0x1, 0xc3ab9c8100, 0xea, 0x100, 0xc3ab14e800, 0xc1, 0x100, 0xc3aba06ba0, 0xc3aba06ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2585 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1440, 0x12f, 0xc36adb1100, 0xec, 0x100, 0xc36e860800, 0xad, 0x100, 0xc36efce540, 0xc36efce480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3625 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21d3d15e0, 0x3b9aca00, 0xc3b51c5a80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3b4f91340, 0x7f7829913900, 0xc3b4f9dd40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f7829913900, 0xc3b4f9dd40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 2246 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75440, 0x0, 0xc33adb8800, 0xea, 0x100, 0xc311188800, 0xb8, 0x100, 0xc343479c80, 0xc343479bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2295 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34fe75440, 0x68, 0xc2dd961c00, 0xec, 0x100, 0xc2c1ee4800, 0xc2, 0x100, 0xc2ed6e79e0, 0xc2ed6e7920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3409 [IO wait]:
net.(*pollDesc).Wait(0xc3c4a7e4c0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c4a7e4c0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c4a7e460, 0xc3c4ef3000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3c60ded18)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3c41a3538, 0xc3c4ef3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc3c41a3538, 0xc3c49dda78, 0xc3c4ef3000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3c4fb8340, 0xc3c4ef3000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c4f05e60)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c4f05e60, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c49dda20)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3604 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066e40, 0x7f782990d810, 0xc3e090c1e0, 0xc3e1cdb2b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7f782990d810, 0xc3e090c1e0, 0xc3e1cdb2b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f782990d810, 0xc3e090c1e0, 0xc3e1cdb2b0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066ea0, 0x7f782990d810, 0xc3e090c1e0, 0xc3e1cdb2b0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3e090c140)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3620 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066e40, 0x7f782990d810, 0xc3e04aa820, 0xc3502de1a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7f782990d810, 0xc3e04aa820, 0xc3502de1a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f782990d810, 0xc3e04aa820, 0xc3502de1a0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066ea0, 0x7f782990d810, 0xc3e04aa820, 0xc3502de1a0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3e04aa640)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2480 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30b9a1440, 0x12f, 0xc220a0e700, 0xec, 0x100, 0xc363d2d800, 0x73, 0x80, 0xc363d7bec0, 0xc363d7be00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2882 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39fd47440, 0x12c, 0xc3a9c12f00, 0xec, 0x100, 0xc3a996b800, 0xb3, 0x100, 0xc3a9e30a20, 0xc3a9e30960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3154 [semacquire]:
sync.(*WaitGroup).Wait(0xc3b4f9dee0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc3b4f9dd40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3b4f9dd40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2997 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066e40, 0x7f782990d810, 0xc3e090c320, 0xc3e0d27380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7f782990d810, 0xc3e090c320, 0xc3e0d27380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f782990d810, 0xc3e090c320, 0xc3e0d27380)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066ea0, 0x7f782990d810, 0xc3e090c320, 0xc3e0d27380)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3af85e460)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3585 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xeb1a20, 0xc3e1fe6240, 0xc3e1fe6270, 0xc2080fa620, 0xc3d95ae070, 0xc2080f00c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:282 +0x6ed
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 3433 [IO wait]:
net.(*pollDesc).Wait(0xc3c4a39f70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c4a39f70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c4a39f10, 0xc3c5560000, 0x1000, 0x1000, 0x0, 0x7f7829909df0, 0xc3c558e4b8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3c41a3598, 0xc3c5560000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f782990bb08, 0xc3c41a3598, 0xc3c4409e98, 0xc3c5560000, 0x1000, 0x1000, 0x7f78298f84c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3c5525c60, 0xc3c5560000, 0x1000, 0x1000, 0xc1ec3e5c83, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c551bf20)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c551bf20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c4409e40)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3434 [select]:
net/http.(*persistConn).writeLoop(0xc3c4409e40)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3639 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3b4f9db00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3b4f9db00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccb2c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3195 [semacquire]:
sync.(*RWMutex).RLock(0xc3be416240)
	/usr/local/go/src/sync/rwmutex.go:36 +0x5f
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3be416240, 0xc3be3c7800)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:553 +0x6f1
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3109 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3b4f76300)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3191 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3be3e3e90, 0xc3be3c6d20, 0xc3be3c6d80)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 2954 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc39fd47440, 0xc8, 0xc3ab9c9500, 0xec, 0x100, 0xc3ac65e000, 0xbb, 0x100, 0xc3ac62bd40, 0xc3ac62bc80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3619 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066e40, 0x7f782990d810, 0xc3e04aa780, 0xc3502de0d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7f782990d810, 0xc3e04aa780, 0xc3502de0d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f782990d810, 0xc3e04aa780, 0xc3502de0d0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066ea0, 0x7f782990d810, 0xc3e04aa780, 0xc3502de0d0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3e04aa5a0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3624 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21d3d1380, 0x3b9aca00, 0xc3b51c5a80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3b4f91340, 0x7f7829913ca8, 0xc3b4f76600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f7829913ca8, 0xc3b4f76600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3650 [chan receive]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccbb00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3426 [select]:
net/http.(*persistConn).writeLoop(0xc3c49dda20)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3432 [select]:
net/http.(*persistConn).writeLoop(0xc3c49dd810)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3644 [chan receive]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc224ccb680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182
[goport] 2017/04/25 13:22:57 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:22:57.774-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:22:57.775-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:22:57.775-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:22:57.840-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:22:57.843-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:22:57.843-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:22:57.843-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:22:57.843-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:22:57.844-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:22:57.844-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:22:57.844-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:22:57.844-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:22:57.844-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:22:57.844-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:22:57.844-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:22:57.844-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:22:57.844-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:22:57.844-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:22:57.844-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:22:57.844-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:22:57.844-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:22:57.844-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:22:57.845-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:22:57.845-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:22:57.848-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:22:57.848-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:22:57.848-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:22:57.848-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:22:58.082-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:23:12.889-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:23:12.899-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:23:12.899-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:23:12.899-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:23:12.900-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:23:12.900-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:23:12.900-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:23:12.900-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:23:12.900-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:23:13.008-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:23:13.008-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:23:13.008-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:23:13.056-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:23:13.057-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:23:13.058-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:23:13.058-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:23:13.058-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:23:13.058-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:23:13.058-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:23:13.059-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:23:13.063-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
GenericSupervisor 2017-04-25T13:23:13.065-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:23:13.065-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
GenericSupervisor 2017-04-25T13:23:13.065-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:13.065-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:13.065-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:13.065-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:13.065-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:23:13.065-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
CheckpointManager 2017-04-25T13:23:13.065-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
CapiUtils 2017-04-25T13:23:13.099-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:23:13.099-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:23:13.100-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:23:13.101-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
StatisticsManager 2017-04-25T13:23:13.101-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:23:13.101-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
XDCRFactory 2017-04-25T13:23:13.101-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:23:13.102-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993 is constructed, start it
GenericPipeline 2017-04-25T13:23:13.103-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[182 222 269 248 338 206 283 310 315 203 268 173 224 243 252 294 207 314 244 340 262 281 300 176 216 198 287 335 188 231 267 179 271 306 319 322 339 205 221 258 296 307 290 229 336 190 215 191 280 324 327 334 217 246 272 288 302 208 233 183 199 266 278 303 329 219 228 234 259 325 214 209 232 256 284 261 277 311 171 242 245 279 282 227 197 291 333 186 237 289 285 337 212 274 275 308 330 184 202 204 321 253 178 220 211 247 292 312 331 192 293 328 223 210 304 299 317 181 241 276 174 213 236 240 251 177 180 194 218 225 175 235 249 264 318 323 189 195 239 273 193 230 301 295 200 260 332 250 297 305 238 257 286 254 263 187 255 265 172 201 320 298 326 226 185 270 313 196 309 316] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[25 67 94 152 163 100 135 12 44 62 88 17 22 39 121 4 150 3 51 96 54 111 140 5 110 116 157 28 166 20 84 26 58 70 125 74 89 119 123 122 145 21 49 60 55 148 95 132 139 138 91 93 126 40 59 153 38 68 83 136 147 161 43 65 164 1 7 30 115 154 112 10 141 8 160 42 81 170 75 98 41 45 50 31 63 56 86 159 32 71 72 9 34 117 146 14 92 137 158 165 169 6 130 155 113 23 77 79 133 97 168 37 80 82 15 19 109 2 18 53 167 13 78 144 114 156 107 16 52 106 142 27 104 105 128 99 108 48 127 162 134 85 87 129 33 36 149 66 73 131 57 76 29 35 103 11 46 64 120 124 24 69 102 0 143 47 118 151 61 90 101]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[242 245 279 282 171 227 197 291 186 333 237 289 337 212 274 285 308 330 184 275 204 202 321 178 253 220 247 292 312 331 211 192 293 328 223 210 304 317 299 241 181 174 213 276 240 251 236 180 194 218 225 175 177 235 249 264 318 323 189 195 239 273 193 230 295 301 260 332 200 297 305 250 257 286 238 263 187 254 265 255 201 172 320 326 226 298 270 313 185 309 316 196 222 269 182 338 206 248 310 315 203 268 283 173 224 243 252 294 207 314 244 340 281 300 176 216 262 198 287 335 231 267 179 188 271 319 322 339 205 221 306 296 307 258 290 336 190 215 229 191 280 324 327 334 217 246 272 288 208 233 302 183 199 266 278 303 329 228 234 259 325 214 219 209 232 256 284 277 311 261] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[38 68 83 136 147 161 65 164 1 7 30 43 115 154 112 10 141 8 160 42 81 170 98 41 45 50 75 31 63 56 86 159 32 71 72 9 34 117 14 92 137 146 165 169 6 130 155 158 23 77 79 113 133 168 37 80 82 97 15 19 109 2 18 53 167 13 78 144 114 156 16 52 106 107 27 104 105 142 128 99 108 48 127 162 85 87 129 134 33 36 149 66 73 131 57 76 29 35 103 11 46 64 124 24 69 102 120 0 143 47 118 151 61 90 101 25 67 94 152 163 100 135 12 44 88 17 22 39 62 121 4 3 51 96 150 54 111 140 5 110 116 157 28 166 20 26 58 70 84 74 89 119 125 123 145 21 49 60 122 55 148 95 132 139 91 93 126 138 40 59 153]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info]
CheckpointManager 2017-04-25T13:23:13.103-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993...
AsyncListener 2017-04-25T13:23:13.114-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:13.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:23:13.115-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:13.115-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:23:13.115-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:23:13.115-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:23:13.115-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:23:13.115-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:23:13.115-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:23:13.115-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:23:13.115-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:23:13.121-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:23:13.121-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:23:13.121-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:23:13.121-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:23:13.125-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
TCPConnPoolMgr 2017-04-25T13:23:13.321-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:23:13.321-05:00 [INFO] connectionSize=5
CapiUtils 2017-04-25T13:23:13.323-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:23:13.323-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:23:13.323-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:23:13.327-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:23:13.329-05:00 [INFO] validateSourceTopology...
TCPConnPoolMgr 2017-04-25T13:23:13.345-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:23:13.345-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:23:13.345-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:23:13.345-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
StatisticsManager 2017-04-25T13:23:13.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:23:13.356-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:23:13.356-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:23:13.415-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:23:13.462-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:23:13.462-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:23:13.462-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:23:13.462-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:23:13.463-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:23:13.463-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:23:13.582-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:23:13.582-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:23:13.599-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:23:13.620-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:23:13.620-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:23:13.873-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:23:13.873-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:23:13.873-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:23:13.873-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:23:13.873-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:23:13.874-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:23:13.874-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993 is started----------
PipelineManager 2017-04-25T13:23:13.874-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:23:13.874-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:23:13.874-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:23:14.194-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:23:15.222-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:23:15.332-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:23:15.332-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:23:15.333-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:23:15.333-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:23:15 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:23:15.488-05:00 [INFO] Send batch count=1
2017/04/25 13:23:15 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:23:15.569-05:00 [INFO] Send batch count=500
2017/04/25 13:23:15 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 112 successful
CapiNozzle 2017-04-25T13:23:15.679-05:00 [INFO] Send batch count=500
2017/04/25 13:23:15 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:23:15 UPR_STREAMREQ for vb 156 successful
CapiNozzle 2017-04-25T13:23:15.936-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:15.957-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:23:16 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 266 successful
CapiNozzle 2017-04-25T13:23:16.170-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:23:16 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 163 successful
CapiNozzle 2017-04-25T13:23:16.215-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
2017/04/25 13:23:16 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 169 successful
StatisticsManager 2017-04-25T13:23:16.272-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:23:16 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 84 successful
CheckpointManager 2017-04-25T13:23:16.449-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
CheckpointManager 2017-04-25T13:23:16.449-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:23:16 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:23:16 UPR_STREAMREQ for vb 98 successful
DcpNozzle 2017-04-25T13:23:16.563-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:23:16.595-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:23:16.654-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:16.665-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:23:16.833-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:23:16.847-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:23:17.018-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:17.033-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
StatisticsManager 2017-04-25T13:23:17.188-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:23:17.208-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:23:17.232-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:23:17.482-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:17.482-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:23:17.482-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:23:17.482-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
ReplicationManager 2017-04-25T13:23:17.482-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
PipelineManager 2017-04-25T13:23:17.489-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:23:17.489-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:23:17.493-05:00 [INFO] err_list=[{"time":"2017-04-25T13:23:17.493002148-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:23:17.493-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:23:17.563-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:17.563-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:17.574-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
CapiNozzle 2017-04-25T13:23:17.722-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:23:17.722-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:23:17.722-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:23:17.722-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:17.722-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:17.807-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:17.807-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:17.833-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:23:17.870-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:23:17.889-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
StatisticsManager 2017-04-25T13:23:18.126-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:23:18.127-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993 {"Errors": "[{\"time\":\"2017-04-25T13:23:17.493002148-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 16, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 633, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 25513, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 633, "set_filtered": 0, "set_received_from_dcp": 25513, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:23:18.290-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:23:18.307-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:23:18.371-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:23:18.392-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
CapiNozzle 2017-04-25T13:23:18.658-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:23:18.678-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:23:18.824-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:23:18.824-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
CapiNozzle 2017-04-25T13:23:18.824-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:23:18.824-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:18.824-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:18.925-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:18.927-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:18.939-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:23:19.135-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:23:19.135-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:23:19.135-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:23:19.135-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:19.135-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:19.196-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:19.196-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:19.210-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:23:19.845-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:19.869-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:23:20.092-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:23:20.125-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:23:20.409-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:20.423-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:23:20.571-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:23:20.587-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:23:20.834-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:20.834-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:23:20.834-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:23:20.834-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:20.834-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:20.953-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:20.953-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:20.969-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:23:20.977-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:23:20.977-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:23:20.977-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:23:20.977-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:20.978-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:21.032-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:21.033-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:21.078-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:23:21.254-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:23:21.265-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:23:21.451-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:21.472-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:23:21.617-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:23:21.636-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:23:21.925-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:21.942-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:23:22.106-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:23:22.107-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:23:22.107-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:23:22.107-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:22.107-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:22.176-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:22.176-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:22.191-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:23:22.534-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:22.534-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:23:22.534-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:23:22.534-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:22.534-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:22.586-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:22.586-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:22.613-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:23:22.986-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:23:23.004-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
GenericSupervisor 2017-04-25T13:23:23.140-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:23:23.140-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:23:23.142-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:23:23.334-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:23:23.356-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
ToplogyChangeDetector 2017-04-25T13:23:23.600-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:23:23.726-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:23:23.745-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:23:24.005-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:23:24.033-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:23:24.220-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:23:24.220-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:23:24.220-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:23:24.220-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:24.220-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:24.306-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:24.306-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:24.339-05:00 [INFO] Retrying update docs for vb 205 for the 1th time
CapiNozzle 2017-04-25T13:23:24.908-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:23:24.908-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:23:24.908-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:23:24.908-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:24.908-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:25.021-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:25.021-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:25.057-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:23:25.321-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:23:25.346-05:00 [INFO] Retrying update docs for vb 205 for the 2th time
CapiNozzle 2017-04-25T13:23:25.530-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:25.544-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:23:25.664-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:23:25.675-05:00 [INFO] Retrying update docs for vb 205 for the 3th time
CapiNozzle 2017-04-25T13:23:25.969-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:25.980-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:23:26.216-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:23:26.216-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 205 after 3 retries
CapiNozzle 2017-04-25T13:23:26.216-05:00 [ERROR] Raise error condition batch update docs failed for vb 205 after 3 retries
GenericSupervisor 2017-04-25T13:23:26.216-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:26.216-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:26.326-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:26.326-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:26.337-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:23:26.562-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:26.563-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:23:26.563-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:23:26.563-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:26.563-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:26.616-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:26.616-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:26.627-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:23:26.664-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:23:26.675-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:23:27.008-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:27.018-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:23:27.220-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:23:27.231-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:23:27.379-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:27.390-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
PipelineManager 2017-04-25T13:23:27.493-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:23:27.493-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:23:27.493-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:23:27.493-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
PipelineRuntimeCtx 2017-04-25T13:23:27.493-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:23:27.493-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:23:27.493-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:23:27.493-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:23:27.493-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:23:27.493-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:23:27.493-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:23:27.493-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:23:27.493-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:27.494-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:23:27.494-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:27.494090076 -0500 CDT
GenericSupervisor 2017-04-25T13:23:27.494-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:27.494-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:23:27.494-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-58348993 has been stopped
GenericPipeline 2017-04-25T13:23:27.494-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-58348993 is stopped
PipelineManager 2017-04-25T13:23:27.494-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:23:27.494-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:23:17.493002148-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:23:27.494-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:23:27.494-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
GenericPipeline 2017-04-25T13:23:27.495-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
GenericPipeline 2017-04-25T13:23:27.495-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
GenericPipeline 2017-04-25T13:23:27.495-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-58348993
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:23:27.495-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:23:27.495-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:23:27.495-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:23:27.495-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:23:27.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:23:27.495-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:23:27.495-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:23:27.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:23:27.496-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:23:27.496-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:27.495748035 -0500 CDT
DcpNozzle 2017-04-25T13:23:27.496-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:27.496100543 -0500 CDT
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:27.496-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:23:27.496-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:23:27.497-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:23:27.497-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:27.497-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:23:27.497-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:27.497-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:23:27.498-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:23:27.498-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:27.498259265 -0500 CDT
DcpNozzle 2017-04-25T13:23:27.498-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:23:27.498-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:27.498-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:23:27.499-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:23:27.499-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:27.499715235 -0500 CDT
DcpNozzle 2017-04-25T13:23:27.499-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:23:27.499-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:27.499-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:23:27 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:23:27 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:23:27 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:23:27 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:23:27 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:23:27 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:23:27 Stream Closed for vb 174 StreamEnd simulated
XDCRFactory 2017-04-25T13:23:27.637-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:23:27.637-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:23:27.637-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:23:27.737-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:23:27.737-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:23:27.737-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:23:27.737-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:23:27.737-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:23:27.737-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:23:27.738-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:23:27.738-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:23:27.738-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
GenericSupervisor 2017-04-25T13:23:27.740-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:23:27.740-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
GenericSupervisor 2017-04-25T13:23:27.740-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:27.740-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:27.740-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:27.740-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:27.740-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:23:27.740-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
CheckpointManager 2017-04-25T13:23:27.740-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
CapiUtils 2017-04-25T13:23:27.815-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:23:27.815-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:23:27.815-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:23:27.817-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
StatisticsManager 2017-04-25T13:23:27.817-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:23:27.817-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
XDCRFactory 2017-04-25T13:23:27.818-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:23:27.818-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375 is constructed, start it
GenericPipeline 2017-04-25T13:23:27.820-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[219 296 189 281 300 317 320 203 243 310 315 318 200 306 307 338 211 263 329 184 206 213 266 289 332 244 291 297 275 240 273 174 214 292 316 293 182 223 237 295 224 323 175 187 229 246 256 279 298 309 271 230 270 334 212 265 245 308 208 228 280 340 180 302 303 179 259 194 282 294 193 257 195 233 262 325 277 322 176 202 236 288 331 269 268 209 222 255 258 238 304 227 241 305 226 285 207 172 183 197 201 313 204 250 253 283 198 216 220 177 221 321 328 299 278 254 312 217 242 276 301 311 326 251 287 192 234 286 335 290 319 324 333 272 181 330 339 218 261 248 231 191 284 314 188 215 235 205 190 196 199 336 264 171 178 260 210 225 337 239 247 327 173 186 274 249 267 252 232 185] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[47 117 165 17 101 170 28 42 84 23 152 151 38 55 57 93 68 113 158 37 40 50 62 14 94 127 163 48 107 5 51 72 161 82 87 121 111 134 0 49 52 54 65 119 22 39 73 150 3 81 136 142 148 11 88 92 29 123 130 164 30 41 53 90 139 143 6 21 27 118 59 106 126 146 149 19 112 144 13 80 169 108 10 95 43 71 9 168 97 25 138 160 132 157 1 16 20 24 79 86 63 8 26 91 154 56 66 129 137 104 18 78 99 115 116 4 131 141 46 69 98 155 12 109 128 133 45 105 122 44 89 67 75 85 35 74 83 7 15 64 145 110 33 61 153 76 162 2 31 140 77 147 58 102 135 156 32 103 60 70 100 159 166 34 36 114 124 96 120 125 167]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[40 50 62 68 113 158 37 94 127 163 14 107 48 51 72 161 5 87 121 82 49 52 54 111 134 0 119 65 39 73 150 22 81 3 142 148 136 88 92 11 123 130 29 164 41 53 90 30 21 27 118 139 143 6 59 126 146 149 106 19 144 112 80 169 13 108 95 10 71 43 168 9 97 138 160 25 16 20 24 132 157 1 86 79 63 8 91 26 66 129 137 154 56 104 18 99 115 116 78 131 141 4 69 98 46 109 128 133 155 12 105 122 45 89 44 75 85 67 74 83 35 15 64 145 7 110 61 153 33 162 76 31 2 140 77 147 102 135 156 58 103 32 70 100 60 36 114 124 159 166 34 120 125 167 96 117 165 47 101 170 17 42 28 84 152 23 55 57 93 151 38] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[211 329 263 184 206 213 266 332 289 291 297 244 240 273 275 174 214 292 316 182 293 223 237 295 175 187 224 323 229 246 256 279 309 298 271 212 230 270 334 265 308 245 180 208 228 280 340 302 303 179 259 194 282 294 193 257 195 233 262 325 176 202 236 277 322 288 269 331 209 222 255 268 258 238 304 227 241 305 207 226 285 172 183 201 313 197 204 250 198 216 220 253 283 177 221 321 328 299 278 254 312 217 242 276 301 326 311 251 287 192 234 286 319 324 333 335 290 181 272 218 261 330 339 231 248 191 284 188 215 235 314 190 196 205 199 336 171 178 260 264 210 225 239 247 327 337 173 186 249 267 274 232 252 185 189 219 296 203 281 300 317 320 243 310 315 200 306 318 307 338]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: worker_batch_size:500 failure_restart_interval:10 target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:23:27.820-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375...
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:27.834-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
CheckpointManager 2017-04-25T13:23:27.834-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:23:27.834-05:00 [INFO] Checkpointing starts in 1802.097 sec
PipelineRuntimeCtx 2017-04-25T13:23:27.834-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:23:27.834-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:23:27.834-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:23:27.836-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:23:27.836-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:23:27.836-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:23:27.836-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:27.836-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:23:27.836-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:23:27.836-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:23:27.838-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:23:27.840-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
StatisticsManager 2017-04-25T13:23:28.180-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:23:28.191-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:23:28.193-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:23:28.193-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:23:28.193-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:23:28.209-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:23:28.247-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:23:28.247-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:23:28.247-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:23:28.273-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:23:28.273-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
PipelineManager 2017-04-25T13:23:28.297-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:23:17.493002148-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={The runtime context is started}
]
CheckpointManager 2017-04-25T13:23:28.302-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:23:28.357-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:23:28.357-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:23:28.357-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:23:28.357-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:23:28.357-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:23:28.357-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:23:28.412-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:23:28.412-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:23:28.412-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:23:28.426-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:23:28.426-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:23:28.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:23:28.576-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:23:28.576-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:23:28.577-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:23:28.577-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:23:28.577-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:23:28.577-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375 is started----------
PipelineManager 2017-04-25T13:23:28.577-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:23:28.577-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:23:28.577-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:23:28.960-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:23:29.950-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:23:30.149-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:23:30.150-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:23:30.150-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:23:30.150-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:23:30 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:23:30.225-05:00 [INFO] Send batch count=172
2017/04/25 13:23:30 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:23:30.299-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:30.299-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:23:30 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:23:30.326-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:23:30.380-05:00 [INFO] Send batch count=91
CapiNozzle 2017-04-25T13:23:30.402-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:30.402-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:23:30 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 304 successful
CapiNozzle 2017-04-25T13:23:30.429-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:23:30 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 41 successful
CapiNozzle 2017-04-25T13:23:30.838-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:30.861-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:23:30.874-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:23:30 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 156 successful
CapiNozzle 2017-04-25T13:23:30.896-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:23:30 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 256 successful
StatisticsManager 2017-04-25T13:23:30.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:23:30 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:23:30 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 289 successful
CheckpointManager 2017-04-25T13:23:31.298-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
CheckpointManager 2017-04-25T13:23:31.298-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:23:31 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:23:31 UPR_STREAMREQ for vb 95 successful
DcpNozzle 2017-04-25T13:23:31.385-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:23:31.460-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:23:31.726-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:23:31.741-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:23:31.882-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:23:31.945-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:31.958-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:23:32.201-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:23:32.202-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:23:32.202-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:23:32.202-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:23:32.202-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:23:32.216-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:23:32.216-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:23:32.218-05:00 [INFO] err_list=[{"time":"2017-04-25T13:23:32.218987449-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:23:32.219-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:23:32.342-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:32.343-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:32.358-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:23:32.769-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:32.769-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:23:32.769-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:23:32.769-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:32.769-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:32.841-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:32.841-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:23:32.845-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:23:32.846-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375 {"Errors": "[{\"time\":\"2017-04-25T13:23:32.218987449-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 117, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 68797, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 117, "set_filtered": 0, "set_received_from_dcp": 68797, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:23:32.852-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:23:32.958-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:23:32.981-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:23:33.568-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:33.587-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:23:33.737-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:23:33.751-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:23:34.275-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:34.287-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:23:34.790-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:23:34.790-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:23:34.790-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:23:34.790-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:34.790-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:34.891-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:34.891-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:34.937-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:23:35.405-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:35.419-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:23:35.419-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:23:35.419-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:35.419-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:35.491-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:35.491-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:35.504-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:23:35.865-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:23:35.881-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:23:36.378-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:23:36.394-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:23:37.191-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:23:37.204-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:23:37.573-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:23:37.586-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
GenericSupervisor 2017-04-25T13:23:37.847-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:23:37.847-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:23:37.847-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:23:38.118-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:23:38.118-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:23:38.118-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:23:38.118-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:38.118-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:38.191-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:38.192-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:38.204-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
ToplogyChangeDetector 2017-04-25T13:23:38.318-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:23:38.690-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:23:38.690-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:23:38.690-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:23:38.690-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:38.690-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:38.771-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:38.772-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:38.783-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:23:39.224-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:23:39.235-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:23:39.615-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:39.626-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:23:39.892-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:23:39.903-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:23:40.145-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:40.156-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:23:40.468-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:23:40.468-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:23:40.468-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:23:40.468-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:40.468-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:40.559-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:40.559-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:40.570-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:23:40.989-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:40.989-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:23:40.989-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:23:40.989-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:40.989-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:41.084-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:41.085-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:41.096-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:23:41.218-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:41.228-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:23:41.869-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:41.880-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:23:42.019-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:42.030-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
PipelineManager 2017-04-25T13:23:42.219-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:23:42.219-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:23:42.219-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:23:42.219-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
PipelineRuntimeCtx 2017-04-25T13:23:42.219-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:23:42.219-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:42.219-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:23:42.219-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:42.219640656 -0500 CDT
GenericSupervisor 2017-04-25T13:23:42.219-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:42.219-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:23:42.219-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:23:42.219-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:23:42.219-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:23:42.219-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:23:42.219-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:23:42.219-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:23:42.219-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:23:42.219-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-738026375 has been stopped
GenericPipeline 2017-04-25T13:23:42.220-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-738026375 is stopped
PipelineManager 2017-04-25T13:23:42.220-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:23:42.220-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:23:32.218987449-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:23:42.220-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:23:42.220-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
GenericPipeline 2017-04-25T13:23:42.220-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
GenericPipeline 2017-04-25T13:23:42.220-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:23:42.220-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-738026375
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:23:42.220-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:23:42.220-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:23:42.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:23:42.221-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:23:42.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:23:42.221-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:23:42.221-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:23:42.221-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:23:42.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:23:42.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:42.221-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:23:42.221-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:42.22103365 -0500 CDT
DcpNozzle 2017-04-25T13:23:42.221-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:23:42.221-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:42.221491461 -0500 CDT
DcpNozzle 2017-04-25T13:23:42.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:23:42.222-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:23:42.222-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:23:42.222-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:42.222-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:23:42.222-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:42.222-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:23:42.222-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:23:42.222-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:23:42.222-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:23:42.222-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:42.222-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:23:42.224-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:23:42.224-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:42.224085749 -0500 CDT
DcpNozzle 2017-04-25T13:23:42.224-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:23:42.224-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:42.224-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:23:42.224-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:23:42.224-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:42.224699589 -0500 CDT
DcpNozzle 2017-04-25T13:23:42.224-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:23:42.224-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:42.224-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:23:42 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:23:42 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:23:42.358-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:23:42.358-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:23:42.359-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:23:42.459-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:23:42.459-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:23:42.460-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:23:42.460-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:23:42.460-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:23:42.460-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:23:42.460-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:23:42.460-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:23:42.461-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
GenericSupervisor 2017-04-25T13:23:42.463-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:23:42.463-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
GenericSupervisor 2017-04-25T13:23:42.463-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:42.463-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:42.463-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:42.463-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:23:42.463-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:23:42.463-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
CheckpointManager 2017-04-25T13:23:42.464-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
CapiUtils 2017-04-25T13:23:42.545-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:23:42.545-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:23:42.545-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:23:42.547-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
StatisticsManager 2017-04-25T13:23:42.547-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:23:42.547-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
XDCRFactory 2017-04-25T13:23:42.548-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:23:42.548-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759 is constructed, start it
GenericPipeline 2017-04-25T13:23:42.550-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[71 47 55 110 120 20 88 156 165 22 66 62 159 53 69 126 135 148 160 170 15 70 86 163 21 2 49 100 166 39 94 138 0 112 130 133 103 137 157 50 107 108 46 115 59 27 79 32 68 75 60 43 31 85 29 101 150 164 116 123 4 92 113 40 134 56 6 19 30 37 65 81 84 128 3 119 136 102 73 80 8 78 26 155 9 48 89 33 23 64 111 131 149 7 42 5 95 52 118 122 14 97 161 91 45 147 38 34 57 18 36 114 12 121 162 169 98 125 67 90 17 24 83 142 154 13 99 58 139 146 61 77 72 127 25 54 124 167 51 74 106 143 63 28 153 16 82 96 76 151 144 44 11 35 41 117 141 158 1 105 132 145 104 168 10 140 93 152 129 109 87] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[261 273 294 313 213 327 332 269 239 245 272 276 308 251 231 223 297 310 311 227 262 264 321 240 268 293 214 189 196 238 173 229 292 319 337 175 178 184 191 252 217 220 259 216 194 329 246 289 290 316 180 199 301 200 203 274 284 195 333 254 266 250 323 212 280 296 181 202 228 270 328 197 257 291 236 267 336 188 218 205 171 320 324 222 226 286 288 304 198 235 331 230 282 303 330 192 242 312 253 183 278 185 224 281 237 317 338 208 193 209 210 314 177 244 283 285 306 176 186 206 249 255 260 334 325 256 318 215 263 225 277 326 271 307 258 298 322 172 182 302 309 340 339 174 232 179 201 241 295 305 221 247 234 243 248 287 299 300 335 187 204 207 315 190 233 211 275 219 265 279]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[321 227 262 264 214 240 268 293 189 196 238 173 229 292 319 337 178 184 191 252 175 220 259 217 216 194 329 290 316 246 289 199 301 180 284 200 203 274 333 195 254 266 323 250 296 212 280 270 328 181 202 228 291 197 257 188 236 267 336 205 218 320 324 171 226 286 288 304 222 198 235 331 282 230 330 303 192 242 253 312 183 278 185 224 281 317 338 237 208 209 210 314 193 177 244 283 285 306 206 249 255 260 176 186 325 334 256 318 215 263 277 326 225 271 307 322 258 298 172 182 302 309 339 340 174 232 201 241 295 305 179 221 247 248 287 234 243 299 300 335 187 204 207 315 190 233 211 275 279 219 265 294 313 261 273 213 327 332 269 245 272 276 308 239 231 251 223 297 310 311] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[2 39 49 100 166 0 94 138 103 112 130 133 50 137 157 46 107 108 115 27 59 32 79 60 68 75 43 29 31 85 101 116 150 164 4 123 40 92 113 134 6 56 65 81 84 128 3 19 30 37 102 119 136 8 73 80 78 9 26 155 33 48 89 131 149 7 23 64 111 5 42 95 14 52 118 122 91 97 161 38 45 147 18 34 57 12 36 114 98 121 162 169 67 125 90 142 154 13 17 24 83 58 99 61 139 146 72 77 25 127 51 54 124 167 63 74 106 143 16 28 153 82 76 96 144 151 44 117 141 158 1 11 35 41 104 105 132 145 10 168 93 140 129 152 87 109 47 71 20 55 110 120 88 22 156 165 66 53 62 159 148 160 170 15 69 126 135 21 70 86 163]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info stats_interval:1000 replication_type:capi checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 filter_expression: active:true worker_batch_size:500]
CheckpointManager 2017-04-25T13:23:42.550-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759...
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:23:42.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:23:42.576-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:42.576-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:23:42.576-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:23:42.576-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:23:42.576-05:00 [INFO] Checkpointing starts in 1802.227 sec
PipelineRuntimeCtx 2017-04-25T13:23:42.577-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:23:42.577-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:23:42.577-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:23:42.577-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:23:42.579-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:23:42.579-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:23:42.579-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:23:42.579-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:23:42.583-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:23:42.608-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:23:42.608-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:23:42.608-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:23:42.629-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:23:42.629-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:23:42.633-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:23:42.633-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:23:42.633-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:23:42.656-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:23:42.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:23:42.674-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:23:42.674-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:23:42.674-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:23:42.675-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:23:42.675-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:23:42.676-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
ToplogyChangeDetector 2017-04-25T13:23:42.688-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:23:42.726-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:23:42.726-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:23:42.734-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:23:42.753-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:23:42.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:23:42.806-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:23:42.957-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:23:42.957-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:23:42.957-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:23:42.988-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:23:42.988-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:23:42.988-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:23:42.988-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759 is started----------
PipelineManager 2017-04-25T13:23:42.988-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:23:42.988-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:23:42.988-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:23:42.999-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:23:42.999-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:23:42.999-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:23:42.999-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
PipelineManager 2017-04-25T13:23:43.085-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:23:43 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:23:43.124-05:00 [INFO] Send batch count=46
CapiNozzle 2017-04-25T13:23:43.138-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:43.150-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:23:43 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 11 successful
CapiNozzle 2017-04-25T13:23:43.227-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:23:43.231-05:00 [INFO] Send batch count=19
CapiNozzle 2017-04-25T13:23:43.250-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:43.251-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:23:43 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:23:43.281-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:23:43 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 14 successful
CapiNozzle 2017-04-25T13:23:43.308-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:23:43 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 213 successful
CapiNozzle 2017-04-25T13:23:43.346-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:23:43 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 24 successful
CapiNozzle 2017-04-25T13:23:43.525-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:23:43 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 325 successful
CapiNozzle 2017-04-25T13:23:43.554-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:23:43.583-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:23:43 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 329 successful
CapiNozzle 2017-04-25T13:23:43.616-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
2017/04/25 13:23:43 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 338 successful
StatisticsManager 2017-04-25T13:23:43.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:23:43 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:23:43.820-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:43.838-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:23:43 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 161 successful
CapiNozzle 2017-04-25T13:23:43.895-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:43.895-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:23:43.895-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:23:43.895-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:23:43.895-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
2017/04/25 13:23:43 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 257 successful
PipelineManager 2017-04-25T13:23:43.907-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:23:43.907-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:43.908-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:43.908-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:23:43.908-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:23:43.908-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:43.908-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:23:43.910-05:00 [INFO] err_list=[{"time":"2017-04-25T13:23:43.910110379-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:23:43.910-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:23:43 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:23:43 UPR_STREAMREQ for vb 69 successful
CapiNozzle 2017-04-25T13:23:44.009-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:44.009-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:44.010-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:44.010-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:23:44 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 174 successful
CapiNozzle 2017-04-25T13:23:44.038-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:23:44.058-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
2017/04/25 13:23:44 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:23:44 UPR_STREAMREQ for vb 294 successful
CheckpointManager 2017-04-25T13:23:44.294-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
CheckpointManager 2017-04-25T13:23:44.294-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:23:44 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:23:44.397-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
2017/04/25 13:23:44 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:23:44.421-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
DcpNozzle 2017-04-25T13:23:44.482-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:23:44.489-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:23:44.550-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:23:44.569-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
StatisticsManager 2017-04-25T13:23:44.593-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:23:44.594-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759 {"Errors": "[{\"time\":\"2017-04-25T13:23:43.910110379-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 26, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 17625, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 26, "set_filtered": 0, "set_received_from_dcp": 17625, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:23:44.712-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:23:44.727-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:23:44.914-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:23:44.933-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:23:45.143-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:23:45.143-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:23:45.143-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:23:45.143-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:45.143-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:45.220-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:45.221-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:45.233-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:23:45.345-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:23:45.345-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:23:45.345-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:23:45.345-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:45.345-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:45.467-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:45.467-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:45.487-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:23:45.601-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:23:45.615-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:23:45.857-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:23:45.916-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:23:46.805-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:23:46.809-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:23:46.869-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:23:46.898-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:23:47.417-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:23:47.417-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:23:47.417-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:23:47.417-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:47.417-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:47.497-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:47.497-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:47.515-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:23:47.861-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:23:47.861-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:23:47.861-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:23:47.861-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:47.861-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:47.939-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:47.940-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:47.961-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:23:48.015-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:48.034-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:23:48.264-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:23:48.285-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:23:48.410-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:48.433-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:23:48.864-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:23:48.878-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:23:49.160-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:23:49.160-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:23:49.160-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:23:49.160-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:49.160-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:49.261-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:49.261-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:49.276-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:23:49.363-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:23:49.363-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:23:49.363-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:23:49.363-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:49.363-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:49.510-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:49.510-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:49.515-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:49.530-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:23:49.543-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:23:50.019-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:50.035-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:23:50.354-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:23:50.381-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:23:50.492-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:23:50.492-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:23:50.492-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:23:50.492-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:50.492-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:50.631-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:50.631-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:50.650-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:23:50.839-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:23:50.870-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:23:51.096-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:51.120-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:23:51.360-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:23:51.370-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:23:51.370-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:23:51.370-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:51.370-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:51.518-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:51.535-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:51.553-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:23:51.590-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:51.663-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:23:51.958-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:23:51.958-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:23:51.958-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:23:51.958-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:51.958-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:52.056-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:52.060-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:52.078-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:23:52.103-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:52.132-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:23:52.516-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:52.532-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
GenericSupervisor 2017-04-25T13:23:52.580-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:23:52.580-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:23:52.580-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:23:52.644-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:23:52.662-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:23:52.887-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:23:52.887-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:23:52.887-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:23:52.887-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:52.887-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:23:52.905-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:23:53.035-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:53.035-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:53.081-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:23:53.103-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:23:53.128-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:23:53.659-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:23:53.659-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:23:53.659-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:23:53.659-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:23:53.659-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:23:53.811-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:23:53.811-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:23:53.832-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:23:53.848-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:23:53.868-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
PipelineManager 2017-04-25T13:23:53.920-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:23:53.920-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:23:53.920-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:23:53.920-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
PipelineRuntimeCtx 2017-04-25T13:23:53.920-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:23:53.920-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:23:53.920-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:23:53.920-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:23:53.920-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:23:53.922-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:23:53.922-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:23:53.922-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:23:53.922-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:53.922-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:23:53.922-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:53.922351794 -0500 CDT
GenericSupervisor 2017-04-25T13:23:53.922-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:23:53.922-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:23:53.922-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-460484759 has been stopped
GenericPipeline 2017-04-25T13:23:53.922-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-460484759 is stopped
PipelineManager 2017-04-25T13:23:53.922-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:23:53.922-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:23:43.910110379-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:23:53.922-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-460484759
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:23:53.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:23:53.923-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:23:53.923-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:23:53.923-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:23:53.923-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:23:53.923-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:23:53.923-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:23:53.924-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:23:53.924-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:23:53.924-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:23:53.924-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:23:53.924-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:53.923817474 -0500 CDT
DcpNozzle 2017-04-25T13:23:53.924-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:23:53.924-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:53.924017645 -0500 CDT
CapiNozzle 2017-04-25T13:23:53.924-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:23:53.924-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:23:53.925-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:23:53.925-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:23:53.925-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:23:53.925-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:53.925-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:23:53.925-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:53.925-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:23:53.927-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:23:53.927-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:23:53.927-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:53.92739753 -0500 CDT
DcpNozzle 2017-04-25T13:23:53.927-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:23:53.927416046 -0500 CDT
DcpNozzle 2017-04-25T13:23:53.927-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:23:53.927-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:23:53.927-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:23:53.927-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.927-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:23:53.927-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:23:53.927-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
2017/04/25 13:23:53 Stream Closed for vb 171 StreamEnd simulated
panic: send on closed channel

goroutine 911 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc26ea0ab40, 0x8edb60, 0xc27b3cdf40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc26eb70f30, 0x8ed920, 0xc29b1b7e40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc26e8d3500, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc2080d8370)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080de001, 0x5ff, 0x5ff, 0xc2080dc080, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080de001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080d4000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4000, 0x8f07a0, 0xc2080664a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc208066480, 0xc2080da5e0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bc00, 0x7f3f3f892cf8, 0xc208066480, 0x0, 0x0, 0xc2080da5e0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bc00, 0x7f3f3f892cf8, 0xc208066480, 0xc208086280, 0xc20801f660, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bc00, 0x7f3f3f892cf8, 0xc208066480)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b3f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b3f90, 0xc2080be480, 0x7f3f3f891480, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 23 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc208084380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 19 [select]:
net/http.(*persistConn).readLoop(0xc20808a0b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 20 [select]:
net/http.(*persistConn).writeLoop(0xc20808a0b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 22 [syscall]:
syscall.Syscall(0x0, 0x0, 0xc20800f000, 0x1000, 0xc20803b1d0, 0xc2081521e0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc20800f000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc208154fa0, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208066b40)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208066b40, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 11 [IO wait]:
net.(*pollDesc).Wait(0xc2080e47d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e47d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e4770, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc27023b030)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038140, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc208038140, 0xc20808a1b8, 0xc2080d3000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801f760, 0xc2080d3000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208066660)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208066660, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a160)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 12 [select]:
net/http.(*persistConn).writeLoop(0xc20808a160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 24 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208084390)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 25 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 26 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 27 [select]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc208082540, 0x11, 0xc20815ffc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc208082540, 0x11, 0xc20815ffc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc208082540, 0x11, 0xc20815ffc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc208154960)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 28 [select]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20802b6c0, 0xf, 0xc20815efc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20802b6c0, 0xf, 0xc20815efc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20802b6c0, 0xf, 0xc20815efc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc208154af0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 29 [select]:
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc208010690)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:205 +0x214
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 30 [select]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208066ae0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:102 +0x75b
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 31 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208066af8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 32 [IO wait]:
net.(*pollDesc).Wait(0xc208010840, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010840, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080107e0, 0x0, 0x7f3f3f890ed8, 0xc289b281d0)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038078, 0xc208164e38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208038078, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208066d20, 0x7f3f3f8947a0, 0xc208038078, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 15 [select]:
net/http.(*persistConn).readLoop(0xc208076000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 16 [select]:
net/http.(*persistConn).writeLoop(0xc208076000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 33 [IO wait]:
net.(*pollDesc).Wait(0xc2080e4450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e4450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e43f0, 0xc208108000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc2080d9c98)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208102050, 0xc208108000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc208102050, 0xc20808a108, 0xc208108000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080da8c0, 0xc208108000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208104360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208104360, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208104360, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf530)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf530, 0xc2080dec00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc20800bc80, 0xc2080dec00, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc20800bc80, 0xc2080dec00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc20800bcc0, 0xc2080dec00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d4180, 0xc2080dcb80, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4180, 0x80f160, 0xc20800be40, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 34 [IO wait]:
net.(*pollDesc).Wait(0xc2080e4c30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e4c30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e4bd0, 0xc20812d000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc2080d9dc0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081020c0, 0xc20812d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc2081020c0, 0xc208076058, 0xc20812d000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080ebd40, 0xc20812d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208104a80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208104a80, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208104a80, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf620)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf620, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc20800bec0, 0xc2080df200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc20800bec0, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc20800bfc0, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d4300, 0xc2080dcd40, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4300, 0x80f160, 0xc20816a080, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 442 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xce, 0xc20802ff00, 0xec, 0x100, 0xc23eeed000, 0xab, 0x100, 0xc23f1bb980, 0xc23f1bb8c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 60 [select]:
net/http.(*persistConn).writeLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 59 [IO wait]:
net.(*pollDesc).Wait(0xc208010b50, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010b50, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010af0, 0xc208168000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc2980f7c68)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038168, 0xc208168000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc208038168, 0xc20808a268, 0xc208168000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081343a0, 0xc208168000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080667e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080667e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 64 [IO wait]:
net.(*pollDesc).Wait(0xc2080e5a30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e5a30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e59d0, 0xc2081c8000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc277c72db8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081020d0, 0xc2081c8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc2081020d0, 0xc2080765d8, 0xc2081c8000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080820e0, 0xc2081c8000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208104d80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208104d80, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076580)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 372 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x12e, 0xc20802f400, 0xec, 0x100, 0xc21b6b1000, 0xb3, 0x100, 0xc21beb6360, 0xc21beb62a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 46 [IO wait]:
net.(*pollDesc).Wait(0xc2080e4f40, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e4f40, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e4ee0, 0xc20812c000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc277a56de8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208102160, 0xc20812c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc208102160, 0xc20808a7e8, 0xc20812c000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080eb740, 0xc20812c000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208105560)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208105560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a790)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 47 [select]:
net/http.(*persistConn).writeLoop(0xc20808a790)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 765 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b566c0, 0x65, 0xc2220caa00, 0xec, 0x100, 0xc25c040800, 0xc6, 0x100, 0xc24fac8180, 0xc24fac80c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 81 [select]:
net/http.(*persistConn).writeLoop(0xc208076580)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1068 [IO wait]:
net.(*pollDesc).Wait(0xc2808c5720, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2808c5720, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2808c56c0, 0xc280ab5000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc281130528)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc27f436d30, 0xc280ab5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc27f436d30, 0xc280ac2898, 0xc280ab5000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2808c38c0, 0xc280ab5000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc280aee780)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc280aee780, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc280ac2840)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1106 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0xc9, 0xc28b4aa900, 0xec, 0x100, 0xc28ab2e800, 0xbd, 0x100, 0xc28bba0de0, 0xc28bba0d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 821 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b56900, 0x12e, 0xc215186d00, 0xec, 0x100, 0xc26d1ec000, 0xb3, 0x100, 0xc26d242300, 0xc26d242240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 418 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xcd, 0xc227aabe00, 0xec, 0x100, 0xc232920000, 0xba, 0x100, 0xc23297f7a0, 0xc23297f6e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1114 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x2, 0xc21969da00, 0xea, 0x100, 0xc2881fb800, 0xcf, 0x100, 0xc2894b2360, 0xc2894b22a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 403 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x4, 0xc215186a00, 0xea, 0x100, 0xc213177000, 0xa2, 0x100, 0xc218dfc0c0, 0xc218dfc000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 356 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x65, 0xc215187200, 0xec, 0x100, 0xc21e16e000, 0xc6, 0x100, 0xc21e2b4060, 0xc21e167f80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 362 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x2, 0xc2220cba00, 0xea, 0x100, 0xc223763800, 0xcf, 0x100, 0xc223cf6fc0, 0xc223cf6f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 316 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x64, 0xc21969c200, 0xec, 0x100, 0xc2195c8000, 0xc9, 0x100, 0xc2195bdf80, 0xc2195bdec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 752 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b56900, 0xca, 0xc24a383300, 0xec, 0x100, 0xc24db52000, 0xbc, 0x100, 0xc24dafbf80, 0xc24dafbec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1036 [IO wait]:
net.(*pollDesc).Wait(0xc27d68f6b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc27d68f6b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc27d68f650, 0xc27d78b000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc29a467ea0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc27d0aa568, 0xc27d78b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc27d0aa568, 0xc27d4d7398, 0xc27d78b000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc27d8274c0, 0xc27d78b000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc27d68d800)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc27d68d800, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc27d4d7340)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1034 [select]:
net/http.(*persistConn).roundTrip(0xc20808a790, 0xc29b8e7370, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc29b52c410, 0xc29b8e1680, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc29b52c410, 0x7f3f3f8910d0, 0xe94600, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc29b52c410, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc29b52c410, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc29b52c410, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc29b52c410, 0x7f3f3f894a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc29b4dd340, 0x9bbca0, 0x6, 0x7f3f3f894a20, 0xc29b8e1640, 0x80f820, 0xc29b8b76f8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc29b8b76e0, 0x9bbca0, 0x6, 0x80f820, 0xc29b8b76f8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc29b8e1620, 0x15, 0x7f3f3f894a20, 0xc29b8e1640, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/goxdcr/utils.LocalPool(0xc29b8e7320, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:85 +0x263
github.com/couchbase/goxdcr/utils.LocalBucket(0xc29b8e7320, 0xe, 0xc20802abf8, 0x8, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:105 +0x166
github.com/couchbase/goxdcr/service_impl.(*ClusterInfoSvc).GetBucket(0xc2080380f0, 0x7f3f3f895070, 0xc20801f440, 0xc20802abf8, 0x8, 0x40adf9, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/cluster_info_service.go:133 +0x25e
github.com/couchbase/goxdcr/factory.(*XDCRFactory).constructSourceNozzles(0xc208152140, 0xc29b57d280, 0xc20800a1c0, 0x32, 0xc29b8e7300, 0x1, 0xc27d7eae10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:280 +0x11c
github.com/couchbase/goxdcr/factory.(*XDCRFactory).NewPipeline(0xc208152140, 0xc20800a1c0, 0x32, 0xc29b8e7210, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:94 +0x39b
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).startPipeline(0xeb0fa0, 0xc20800a1c0, 0x32, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:280 +0x413
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc27cf74c80, 0xc27d68d7a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:593 +0xdc2
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc27cf74c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 638 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b56900, 0x12c, 0xc227aabd00, 0xec, 0x100, 0xc236572c00, 0x2c, 0x40, 0xc230e4c240, 0xc230e4c180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 124 [select]:
net/http.(*persistConn).writeLoop(0xc2080764d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 123 [IO wait]:
net.(*pollDesc).Wait(0xc2080e5f70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e5f70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e5f10, 0xc20831b000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc2774c9c60)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038280, 0xc20831b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc208038280, 0xc208076528, 0xc20831b000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208408ca0, 0xc20831b000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081c6ea0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081c6ea0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080764d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 145 [syscall, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 826 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b566c0, 0x2, 0xc25341c500, 0xea, 0x100, 0xc26d52f000, 0xcf, 0x100, 0xc26d984480, 0xc26d9843c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 431 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x1, 0xc23f868700, 0xea, 0x100, 0xc23eb56800, 0xc7, 0x100, 0xc23f8a4d80, 0xc23f8a4cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 796 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b566c0, 0x1, 0xc22d88f600, 0xea, 0x100, 0xc26b63f000, 0xc7, 0x100, 0xc26ba652c0, 0xc26ba65200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 973 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x1, 0xc277c99300, 0xe9, 0x100, 0xc277e5c000, 0x1e, 0x20, 0xc277e901e0, 0xc277e90120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 388 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xcb, 0xc20802f200, 0xec, 0x100, 0xc222914800, 0xc9, 0x100, 0xc222ff5740, 0xc222ff5680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 437 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x0, 0xc208174e00, 0xea, 0x100, 0xc237653800, 0xb8, 0x100, 0xc237fb99e0, 0xc237fb9920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 754 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b566c0, 0x0, 0xc24a383600, 0xea, 0x100, 0xc2514fb000, 0xa2, 0x100, 0xc251c554a0, 0xc251c553e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1217 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc26e8d3500, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc26e8d3500, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:283 +0x431
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc29b8b7a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1130 [IO wait]:
net.(*pollDesc).Wait(0xc25f5933a0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc25f5933a0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc25f593340, 0xc26e0e9000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc2988c0d80)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc263163758, 0xc26e0e9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc260b86ae8, 0xc26e0e9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc2583ad5e0, 0xc26e0e9000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc26e070720)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc26e070720, 0xc2730c9b0a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc26e070720, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc259d25f50, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc259d25f50, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc26e070720, 0xc29886c9c0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc260b86aa0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc260b86aa0)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 396 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x130, 0xc22d88e900, 0xec, 0x100, 0xc22d543000, 0xbc, 0x100, 0xc22dcd0fc0, 0xc22dcd0f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 344 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xcf, 0xc215187c00, 0xec, 0x100, 0xc217840000, 0xac, 0x100, 0xc21779bc20, 0xc21779bb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1116 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0x12c, 0xc27e3ba100, 0xec, 0x100, 0xc26edbb800, 0xc4, 0x100, 0xc267ae8ea0, 0xc267ae8de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 647 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b566c0, 0x0, 0xc23f869700, 0xea, 0x100, 0xc23e51ec00, 0x66, 0x80, 0xc2341cc7e0, 0xc2341cc720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 542 [IO wait]:
net.(*pollDesc).Wait(0xc232233c60, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc232233c60, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc232233c00, 0xc233aa2000, 0x1000, 0x1000, 0x0, 0x7f3f3f890ed8, 0xc273198ec8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc24047c2b8, 0xc233aa2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f3f3f892bf0, 0xc24047c2b8, 0xc2406992e8, 0xc233aa2000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc21efa1dc0, 0xc233aa2000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc232bd3800)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc232bd3800, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc240699290)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 543 [select]:
net/http.(*persistConn).writeLoop(0xc240699290)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 771 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b56900, 0x12d, 0xc252467100, 0xec, 0x100, 0xc24a822000, 0xcf, 0x100, 0xc24a46d140, 0xc24a46d080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 779 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240b56900, 0x12f, 0xc23f869300, 0xec, 0x100, 0xc268200800, 0xce, 0x100, 0xc26afe86c0, 0xc26afe8600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1127 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0x12d, 0xc2784b3100, 0xec, 0x100, 0xc282cf6800, 0xcf, 0x100, 0xc2667d2120, 0xc2667d2060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1037 [select]:
net/http.(*persistConn).writeLoop(0xc27d4d7340)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1086 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x3, 0xc285a72800, 0xea, 0x100, 0xc288ec4800, 0xb5, 0x100, 0xc289b74600, 0xc289b74540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 908 [runnable]:
sync.(*WaitGroup).Wait(0xc26e8d35f0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*DcpNozzle).onExit(0xc26e8d3500)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:456 +0x37
github.com/couchbase/goxdcr/parts.*DcpNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:153 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc26e8d3500)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1077 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0x12f, 0xc27d20b400, 0xec, 0x100, 0xc27e2a6000, 0xce, 0x100, 0xc27e60b620, 0xc27e60b560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1064 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x5, 0xc27e3bac00, 0xea, 0x100, 0xc27e2a7800, 0xbc, 0x100, 0xc27e7bb320, 0xc27e7bb260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 982 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0xc8, 0xc2784b2800, 0xea, 0x100, 0xc2785bc000, 0x9, 0x10, 0xc27858faa0, 0xc27858f9e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1125 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x1, 0xc2784b2100, 0xea, 0x100, 0xc285c5e800, 0xb3, 0x100, 0xc2199ff680, 0xc2199ff5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1069 [select]:
net/http.(*persistConn).writeLoop(0xc280ac2840)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1201 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc29b8e18c0, 0x3b9aca00, 0xc26eb49b00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc26e494d10, 0x7f3f3f89af28, 0xc26e8d3500, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f3f3f89af28, 0xc26e8d3500)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 1152 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x0, 0xc25341db00, 0xea, 0x100, 0xc23b491000, 0xb8, 0x100, 0xc2784ab9e0, 0xc2784ab920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1160 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0x12e, 0xc22d88eb00, 0xec, 0x100, 0xc26cda0000, 0xb3, 0x100, 0xc26cdcede0, 0xc26cdced20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1166 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x67, 0xc227aaa200, 0xec, 0x100, 0xc292b8c000, 0xb3, 0x100, 0xc292c4cb40, 0xc292c4ca80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1193 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0a900, 0x64, 0xc29a444b00, 0xec, 0x100, 0xc29a2ee800, 0xc9, 0x100, 0xc29ae96a80, 0xc29ae969c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1174 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ea0ab40, 0xc8, 0xc28b4ab100, 0xec, 0x100, 0xc295a4d000, 0xb7, 0x100, 0xc296e5d9e0, 0xc296e5d920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31
[goport] 2017/04/25 13:23:54 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:23:54.475-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:23:54.475-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:23:54.475-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:23:54.497-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:23:54.501-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:23:54.501-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:23:54.501-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:23:54.501-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:23:54.501-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:23:54.502-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:23:54.502-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:23:54.502-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:23:54.502-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
RemoteClusterChangeListener 2017-04-25T13:23:54.502-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationSpecChangeListener 2017-04-25T13:23:54.502-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
ReplicationManager 2017-04-25T13:23:54.502-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:23:54.502-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:23:54.502-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:23:54.502-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:23:54.502-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:23:54.502-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:23:54.503-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:23:54.503-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:23:54.503-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:23:54.508-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:23:54.508-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:23:54.508-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:23:54.508-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:23:55.401-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:24:09.559-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:24:09.562-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:24:09.562-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:24:09.563-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:24:09.562-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:24:09.563-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:24:09.563-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:24:09.563-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:24:09.563-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:24:09.671-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:09.672-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:24:09.672-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:24:09.728-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:09.729-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:24:09.729-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:09.729-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:24:09.729-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:09.729-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:24:09.729-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:24:09.732-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:24:09.735-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
GenericSupervisor 2017-04-25T13:24:09.736-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:24:09.736-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
GenericSupervisor 2017-04-25T13:24:09.736-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:09.736-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:09.736-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:09.736-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:09.736-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:24:09.736-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
CheckpointManager 2017-04-25T13:24:09.736-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
CapiUtils 2017-04-25T13:24:09.766-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:09.766-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:09.766-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:24:09.767-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
StatisticsManager 2017-04-25T13:24:09.767-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:24:09.767-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
XDCRFactory 2017-04-25T13:24:09.767-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:24:09.767-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579 is constructed, start it
GenericPipeline 2017-04-25T13:24:09.768-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[12 4 33 151 91 104 8 50 58 68 74 81 7 40 67 136 156 27 34 98 153 165 1 18 49 99 142 154 10 51 111 123 138 36 76 96 157 0 149 15 147 162 164 22 24 110 143 44 69 71 108 28 124 168 77 112 9 167 63 118 129 140 14 20 57 66 133 11 79 85 101 135 113 32 144 39 102 122 35 117 160 5 89 148 13 56 64 2 126 6 16 119 145 41 60 61 26 131 155 82 132 161 139 73 87 150 47 75 170 25 46 30 159 37 54 70 100 163 121 97 29 42 120 53 83 78 94 95 114 72 107 158 166 169 45 43 115 116 127 146 93 103 152 17 80 84 88 105 128 134 52 65 125 21 90 48 92 137 19 23 55 106 141 59 62 86 3 31 38 109 130] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[257 293 331 241 324 179 192 271 283 329 202 251 278 328 246 261 230 184 185 190 228 269 294 176 254 174 198 222 334 339 258 295 300 221 207 232 299 210 323 182 206 276 290 186 259 273 181 225 233 310 311 312 187 279 309 191 255 188 201 260 265 330 214 317 175 218 263 204 239 177 208 305 242 248 285 173 209 216 226 227 240 268 197 220 326 307 262 272 282 195 224 288 308 301 336 337 266 267 229 253 284 316 333 340 217 252 275 292 203 244 338 231 314 281 277 298 315 319 223 234 238 247 325 205 313 200 215 289 211 219 304 306 193 270 287 303 235 322 335 237 332 178 250 286 291 327 183 189 256 264 180 196 302 318 249 297 199 194 213 245 171 320 280 321 172 212 236 296 243 274]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[315 277 298 234 238 247 319 223 325 205 200 215 289 313 211 219 304 270 287 303 306 193 322 235 237 332 335 178 286 291 327 250 189 256 264 183 180 196 302 318 249 297 194 199 213 245 171 320 280 321 172 212 236 296 243 274 293 331 257 241 324 179 283 329 192 271 251 278 328 202 246 261 230 190 228 184 185 269 294 176 254 174 334 198 222 339 258 295 300 207 221 210 232 299 182 206 276 323 186 259 273 290 225 181 233 310 187 279 309 311 312 191 255 188 201 260 265 330 214 317 175 218 263 204 239 208 305 177 209 216 226 242 248 285 173 197 220 227 240 268 326 307 195 224 262 272 282 288 308 266 267 301 336 337 229 253 284 217 316 333 340 203 244 252 275 292 231 314 338 281] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[139 150 73 87 170 47 75 25 46 30 159 70 100 163 37 54 121 97 120 29 42 53 83 95 114 78 94 158 166 72 107 169 45 116 127 146 43 115 152 93 103 84 88 17 80 134 105 128 125 52 65 21 90 137 48 92 55 106 141 19 23 86 59 62 38 109 130 3 31 12 151 4 33 58 68 74 91 104 8 50 81 67 136 156 7 40 98 27 34 153 165 1 99 18 49 111 123 138 142 154 10 51 96 157 36 76 0 149 162 164 15 147 110 143 22 24 44 69 71 108 28 124 168 77 112 9 167 129 63 118 57 66 133 140 14 20 85 101 135 11 79 113 32 144 122 39 102 160 35 117 148 5 89 64 13 56 2 126 119 145 6 16 61 41 60 155 26 131 161 82 132]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 replication_type:capi active:true source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:24:09.768-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579...
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:09.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
StatisticsManager 2017-04-25T13:24:09.776-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:24:09.777-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:24:09.778-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:24:09.778-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:24:09.778-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:09.778-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:24:09.778-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:24:09.778-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:24:09.778-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:24:09.778-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:24:09.778-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:24:09.779-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:24:09.779-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:24:09.780-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
TCPConnPoolMgr 2017-04-25T13:24:09.901-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:24:09.901-05:00 [INFO] connectionSize=5
StatisticsManager 2017-04-25T13:24:09.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:24:09.911-05:00 [INFO] validateSourceTopology...
TCPConnPoolMgr 2017-04-25T13:24:09.911-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:24:09.911-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:24:09.911-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:24:09.911-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:24:09.919-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:09.919-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:09.919-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:24:09.920-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:24:09.920-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:24:09.936-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:24:09.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:24:09.964-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:09.964-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:24:09.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:24:09.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:24:09.965-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:24:10.079-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:24:10.079-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:24:10.079-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:24:10.101-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:24:10.101-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:24:10.129-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:24:10.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:24:10.221-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:10.221-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:24:10.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:24:10.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:24:10.222-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:24:10.222-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579 is started----------
PipelineManager 2017-04-25T13:24:10.222-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:24:10.222-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:24:10.222-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:24:10.489-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:24:10.490-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:24:10.490-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:24:10.490-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:24:10 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:24:10.703-05:00 [INFO] Send batch count=164
CapiNozzle 2017-04-25T13:24:10.709-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:24:10.715-05:00 [INFO] Send batch count=41
2017/04/25 13:24:10 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 307 successful
CapiNozzle 2017-04-25T13:24:10.791-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
2017/04/25 13:24:10 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 111 successful
CapiNozzle 2017-04-25T13:24:10.829-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
2017/04/25 13:24:10 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 119 successful
StatisticsManager 2017-04-25T13:24:10.917-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:24:10 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:24:10 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 132 successful
CapiNozzle 2017-04-25T13:24:11.084-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:24:11.086-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:24:11 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 230 successful
CapiNozzle 2017-04-25T13:24:11.104-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:24:11.107-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:24:11 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 142 successful
CapiNozzle 2017-04-25T13:24:11.236-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
2017/04/25 13:24:11 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 244 successful
CapiNozzle 2017-04-25T13:24:11.260-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
2017/04/25 13:24:11 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 263 successful
CapiNozzle 2017-04-25T13:24:11.367-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:24:11 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 57 successful
CapiNozzle 2017-04-25T13:24:11.391-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:24:11.391-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:24:11.391-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:24:11.391-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries]
ReplicationManager 2017-04-25T13:24:11.391-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries]
PipelineManager 2017-04-25T13:24:11.410-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:24:11.411-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:11.413-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
PipelineManager 2017-04-25T13:24:11.432-05:00 [INFO] err_list=[{"time":"2017-04-25T13:24:11.432215337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries"}]
PipelineManager 2017-04-25T13:24:11.432-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:24:11 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 75 successful
CapiNozzle 2017-04-25T13:24:11.535-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:11.535-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:24:11 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 284 successful
CapiNozzle 2017-04-25T13:24:11.560-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
2017/04/25 13:24:11 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 197 successful
CheckpointManager 2017-04-25T13:24:11.781-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
CheckpointManager 2017-04-25T13:24:11.781-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:24:11 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:24:11 UPR_STREAMREQ for vb 94 successful
StatisticsManager 2017-04-25T13:24:11.790-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:24:11.791-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579 {"Errors": "[{\"time\":\"2017-04-25T13:24:11.432215337-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 66, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 3178, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 66, "set_filtered": 0, "set_received_from_dcp": 3178, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:24:11.799-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:11.821-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
DcpNozzle 2017-04-25T13:24:11.833-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:24:11.868-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:24:12.025-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:12.042-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:24:12.093-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:12.094-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:24:12.094-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:24:12.094-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:12.094-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:12.147-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:12.147-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:12.181-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:24:12.318-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:12.367-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:24:12.762-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:12.773-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:24:12.975-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:12.975-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:24:12.975-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:24:12.975-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:12.975-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:13.043-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:13.043-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:13.064-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:24:13.180-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:13.222-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:24:13.410-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:24:13.429-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:24:13.592-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:13.592-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:24:13.592-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:24:13.592-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:13.592-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:13.655-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:13.656-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:13.675-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:24:13.745-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:24:13.770-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:24:14.100-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:14.128-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:24:14.247-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:24:14.247-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:24:14.247-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:24:14.247-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:14.247-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:14.341-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:14.342-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:14.356-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:24:14.433-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:14.451-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:24:14.732-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:14.743-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:24:14.958-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:14.958-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:24:14.958-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:24:14.958-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:14.959-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:15.023-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:15.023-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:15.041-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:24:15.165-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:15.178-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:24:15.697-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:15.715-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:24:15.987-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:15.987-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:24:15.987-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:24:15.987-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:15.987-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:16.051-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:16.051-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:16.079-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:24:16.179-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:16.219-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:24:16.635-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:24:16.648-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:16.648-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:24:16.648-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:24:16.648-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:16.648-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:16.658-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:24:16.759-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:16.759-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:16.782-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:24:17.015-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:24:17.036-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:24:17.177-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:24:17.209-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:24:17.472-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:24:17.472-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:24:17.472-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:24:17.472-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:17.472-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:17.527-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:17.528-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:17.561-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:24:17.581-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:24:17.607-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:24:18.247-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:18.259-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:24:18.521-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:24:18.521-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:24:18.521-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:24:18.521-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:18.521-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:18.583-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:18.583-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:18.601-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:24:18.760-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:18.780-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:24:18.833-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:18.851-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:24:19.127-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:19.127-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:24:19.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:24:19.127-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:19.127-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:19.189-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:19.199-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:19.200-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:19.200-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:24:19.211-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:24:19.449-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:19.449-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:24:19.449-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:24:19.449-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:19.449-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:19.534-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:19.534-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:19.545-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:24:19.741-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:24:19.752-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
GenericSupervisor 2017-04-25T13:24:19.779-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:24:19.779-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:24:19.779-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:24:19.903-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:24:19.914-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:24:20.143-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
ToplogyChangeDetector 2017-04-25T13:24:20.151-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:24:20.154-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:24:20.377-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:24:20.387-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:24:20.619-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:24:20.619-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:24:20.619-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:24:20.619-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:20.619-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:20.698-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:20.699-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:20.709-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:24:20.762-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:24:20.763-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:24:20.763-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:24:20.763-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:20.763-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:20.840-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:20.840-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:20.851-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:24:21.141-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:24:21.152-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:24:21.379-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:24:21.393-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
PipelineManager 2017-04-25T13:24:21.443-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries 
PipelineManager 2017-04-25T13:24:21.443-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:24:21.443-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:24:21.443-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
PipelineRuntimeCtx 2017-04-25T13:24:21.443-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:24:21.443-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:21.443-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:24:21.443-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:21.443595909 -0500 CDT
GenericSupervisor 2017-04-25T13:24:21.443-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:21.443-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:24:21.443-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:24:21.443-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:24:21.443-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:24:21.443-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:24:21.443-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:24:21.443-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:24:21.443-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:24:21.443-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-729753579 has been stopped
GenericPipeline 2017-04-25T13:24:21.444-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-729753579 is stopped
PipelineManager 2017-04-25T13:24:21.444-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:24:21.444-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:11.432215337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:24:21.444-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:24:21.445-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
GenericPipeline 2017-04-25T13:24:21.445-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
GenericPipeline 2017-04-25T13:24:21.445-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
GenericPipeline 2017-04-25T13:24:21.445-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-729753579
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:21.445-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:21.446-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:24:21.446-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:24:21.446-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:24:21.446-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:21.446-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:24:21.446-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:24:21.446-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:24:21.446-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:21.446-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:21.446-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:24:21.446-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:21.446395556 -0500 CDT
DcpNozzle 2017-04-25T13:24:21.447-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:24:21.447-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:21.446770678 -0500 CDT
CapiNozzle 2017-04-25T13:24:21.447-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:24:21.447-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:24:21.447-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:24:21.447-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:24:21.447-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.447-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:21.447-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.447-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:21.448-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:24:21.448-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:21.448-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:24:21.451-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:24:21.451-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:24:21.451-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:21.451314214 -0500 CDT
DcpNozzle 2017-04-25T13:24:21.451-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:21.451329524 -0500 CDT
DcpNozzle 2017-04-25T13:24:21.451-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:24:21.451-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:24:21.451-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:21.451-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:24:21.451-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:21.451-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:24:21 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:24:21 Stream Closed for vb 171 StreamEnd simulated
CapiNozzle 2017-04-25T13:24:21.462-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:24:21.462-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:24:21.463-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:21.463-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:24:22.206-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:22.206-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:24:22.206-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:24:22.254-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:22.254-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:24:22.254-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:22.254-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:24:22.254-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:22.254-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:24:22.255-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:24:22.255-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:24:22.255-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
GenericSupervisor 2017-04-25T13:24:22.257-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:24:22.257-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
GenericSupervisor 2017-04-25T13:24:22.257-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:22.257-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:22.257-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:22.257-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:22.257-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:24:22.257-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
CheckpointManager 2017-04-25T13:24:22.257-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
CapiUtils 2017-04-25T13:24:22.283-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:22.283-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:22.283-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:24:22.284-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
StatisticsManager 2017-04-25T13:24:22.284-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:24:22.284-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
XDCRFactory 2017-04-25T13:24:22.284-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:24:22.284-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258 is constructed, start it
GenericPipeline 2017-04-25T13:24:22.285-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[51 56 164 107 167 169 36 157 165 54 59 88 62 3 170 43 109 130 70 104 75 84 141 1 28 32 18 67 148 101 49 99 100 77 90 8 45 60 97 154 27 81 150 38 52 121 127 163 0 7 126 124 159 92 162 68 151 20 30 46 89 131 103 143 22 78 83 138 149 55 66 94 102 11 91 115 120 128 142 4 96 153 17 87 95 6 137 125 9 13 93 47 113 119 158 61 71 116 168 63 117 136 25 98 12 79 86 29 73 76 139 140 145 16 19 53 105 114 57 112 24 26 31 129 80 82 144 166 72 85 152 15 41 69 21 156 74 111 5 23 50 133 2 33 110 42 44 161 35 37 106 118 135 147 14 48 65 146 58 108 132 155 10 122 134 39 123 64 160 34 40] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[189 196 207 305 332 289 334 268 171 264 281 338 252 211 261 319 200 270 288 323 198 228 330 184 220 217 282 182 172 209 272 312 173 226 336 199 313 290 298 230 274 221 328 176 178 296 299 325 194 219 249 223 238 222 283 308 218 242 280 267 275 278 174 302 177 205 248 315 293 339 183 262 309 294 326 191 210 314 331 240 287 297 212 231 235 259 295 335 227 340 188 322 324 255 271 284 192 180 187 233 260 213 266 206 237 250 277 316 204 300 236 256 310 181 306 245 303 224 229 234 307 185 276 179 201 251 265 321 327 232 239 195 329 193 197 292 190 216 243 244 253 257 286 254 263 311 285 317 333 186 258 291 175 214 215 208 269 318 304 301 202 225 246 241 247 273 320 203 337 279]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[34 40 51 56 164 107 167 169 36 157 54 59 88 165 62 3 170 43 109 130 70 104 1 28 32 75 84 141 18 67 148 49 99 100 101 8 45 60 77 90 97 27 81 150 154 38 52 121 127 0 7 126 163 124 159 92 162 68 151 20 30 46 89 131 22 78 83 103 143 55 66 94 138 149 102 11 91 115 120 128 142 4 17 87 95 96 153 6 137 9 13 93 125 47 113 61 71 116 119 158 63 117 136 168 25 98 12 79 86 29 73 76 16 19 53 139 140 145 105 114 24 26 31 57 112 129 80 82 144 166 15 41 69 72 85 152 21 156 5 23 50 74 111 2 33 110 133 42 44 161 35 37 14 48 65 106 118 135 147 58 108 132 146 10 122 134 155 39 123 64 160] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[189 196 207 305 332 268 289 334 338 171 264 281 252 200 211 261 319 270 288 323 198 184 220 228 330 282 217 182 312 172 209 272 336 173 226 199 313 230 274 290 298 221 328 299 176 178 296 194 325 219 249 222 223 238 218 283 308 242 280 174 267 275 278 302 248 177 205 315 293 339 183 262 309 191 210 294 326 314 331 212 240 287 297 295 231 235 259 335 227 188 340 255 271 284 322 324 192 260 180 187 233 206 213 266 237 250 277 316 204 300 256 236 310 181 306 224 245 303 307 229 234 185 276 321 179 201 251 265 327 232 239 329 195 190 193 197 292 244 216 243 253 257 286 254 263 311 285 317 333 186 258 291 175 214 215 208 269 318 301 304 202 225 246 320 241 247 273 203 279 337]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: active:true checkpoint_interval:1800 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:24:22.285-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258...
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:22.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
CheckpointManager 2017-04-25T13:24:22.297-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:24:22.298-05:00 [INFO] Checkpointing starts in 1804.084 sec
PipelineRuntimeCtx 2017-04-25T13:24:22.298-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:24:22.298-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:24:22.298-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:24:22.298-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:24:22.298-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:24:22.298-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:24:22.298-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:22.298-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:24:22.298-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:24:22.299-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:24:22.299-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:24:22.305-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:24:22.318-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:22.318-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:22.318-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:24:22.708-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:24:22.736-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:24:22.736-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:24:22.736-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:24:22.745-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:24:22.748-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:24:22.754-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:24:22.754-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:24:22.807-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:24:22.807-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:22.807-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:24:22.807-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:24:22.807-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:24:22.808-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:24:22.838-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:24:22.921-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:24:22.921-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:24:22.921-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:24:22.939-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:24:22.939-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:24:23.070-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:24:23.070-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:24:23.070-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:24:23.071-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:24:23.113-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:24:23.113-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:23.113-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:24:23.114-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:24:23.114-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:24:23.114-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:24:23.114-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258 is started----------
PipelineManager 2017-04-25T13:24:23.114-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:24:23.114-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:24:23.114-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:24:23 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:24:23.144-05:00 [INFO] Send batch count=141
CapiNozzle 2017-04-25T13:24:23.191-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:23.192-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:23.216-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:24:23 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 316 successful
StatisticsManager 2017-04-25T13:24:23.364-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:24:23 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 111 successful
CapiNozzle 2017-04-25T13:24:23.375-05:00 [INFO] Send batch count=60
2017/04/25 13:24:23 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 143 successful
CapiNozzle 2017-04-25T13:24:23.668-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:23.699-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:24:23 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 272 successful
CapiNozzle 2017-04-25T13:24:23.924-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:23.939-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:24:23 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:24:23 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:24:24 UPR_STREAMREQ for vb 194 successful
CheckpointManager 2017-04-25T13:24:24.026-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
CheckpointManager 2017-04-25T13:24:24.026-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:24:24.108-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:24:24 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:24:24.143-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:24.143-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:24:24.143-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:24:24.143-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:24:24.143-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:24:24.151-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:24:24.151-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:24:24.154-05:00 [INFO] err_list=[{"time":"2017-04-25T13:24:24.154188726-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:24:24.154-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:24:24.211-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:24.211-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
DcpNozzle 2017-04-25T13:24:24.231-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:24:24.231-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
StatisticsManager 2017-04-25T13:24:24.303-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:24:24.304-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258 {"Errors": "[{\"time\":\"2017-04-25T13:24:24.154188726-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 55, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 3721, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 55, "set_filtered": 0, "set_received_from_dcp": 3721, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:24:24.615-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
PipelineManager 2017-04-25T13:24:24.618-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:24.154188726-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:24:24.634-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:24:24.988-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:25.019-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:24:25.408-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:25.408-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:24:25.408-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:24:25.408-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:25.408-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:25.471-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:25.471-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:25.488-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:24:25.985-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:26.011-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:24:26.456-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:26.473-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:24:26.885-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:26.885-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:24:26.885-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:24:26.885-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:26.885-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:26.952-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:26.952-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:26.971-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:24:27.391-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:27.438-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:24:27.686-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:27.702-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:24:27.914-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:27.915-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:24:27.915-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:24:27.915-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:27.915-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:27.934-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 110
CapiNozzle 2017-04-25T13:24:28.646-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:28.647-05:00 [ERROR] batchUpdateDocs for vb 110 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:28.704-05:00 [INFO] Retrying update docs for vb 110 for the 1th time
CapiNozzle 2017-04-25T13:24:29.335-05:00 [ERROR] batchUpdateDocs for vb 110 failed with err receieved unexpected status code, 500, from update docs request for vb 110
.
CapiNozzle 2017-04-25T13:24:29.351-05:00 [INFO] Retrying update docs for vb 110 for the 2th time
CapiNozzle 2017-04-25T13:24:30.310-05:00 [ERROR] batchUpdateDocs for vb 110 failed with err receieved unexpected status code, 500, from update docs request for vb 110
.
CapiNozzle 2017-04-25T13:24:30.329-05:00 [INFO] Retrying update docs for vb 110 for the 3th time
CapiNozzle 2017-04-25T13:24:30.936-05:00 [ERROR] batchUpdateDocs for vb 110 failed with err receieved unexpected status code, 500, from update docs request for vb 110
.
CapiNozzle 2017-04-25T13:24:30.936-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 110 after 3 retries
CapiNozzle 2017-04-25T13:24:30.936-05:00 [ERROR] Raise error condition batch update docs failed for vb 110 after 3 retries
GenericSupervisor 2017-04-25T13:24:30.936-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:30.936-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:30.987-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:30.987-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:31.002-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:24:31.358-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:24:31.369-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:24:31.732-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:24:31.742-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:24:32.127-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:24:32.127-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:24:32.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:24:32.127-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:32.127-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:32.199-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:32.199-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:32.210-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
GenericSupervisor 2017-04-25T13:24:32.300-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:24:32.300-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:24:32.300-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:24:32.513-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:24:32.524-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:24:32.858-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:24:32.868-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
ToplogyChangeDetector 2017-04-25T13:24:33.000-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:24:33.183-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:24:33.183-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:24:33.183-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:24:33.183-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:33.183-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:33.267-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:33.267-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:33.278-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:24:33.462-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:24:33.473-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:24:33.727-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:33.740-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:24:33.766-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:24:33.780-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:24:33.868-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:24:33.879-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:24:34.018-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:34.029-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:24:34.060-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:24:34.060-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:24:34.061-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:24:34.061-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:34.061-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:24:34.154-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:24:34.154-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:24:34.154-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:24:34.154-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
PipelineRuntimeCtx 2017-04-25T13:24:34.154-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:24:34.154-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:24:34.154-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:24:34.154-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:24:34.154-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:24:34.154-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:24:34.154-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:24:34.154-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:24:34.154-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:34.155-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:24:34.155-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:34.155055828 -0500 CDT
GenericSupervisor 2017-04-25T13:24:34.155-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:34.155-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:24:34.155-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-255001258 has been stopped
GenericPipeline 2017-04-25T13:24:34.155-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-255001258 is stopped
PipelineManager 2017-04-25T13:24:34.155-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:24:34.155-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:24.154188726-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:24:34.155-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:24:34.156-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
GenericPipeline 2017-04-25T13:24:34.156-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
GenericPipeline 2017-04-25T13:24:34.156-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
GenericPipeline 2017-04-25T13:24:34.156-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-255001258
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:34.156-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:24:34.156-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:34.156-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:24:34.157-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:24:34.157-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:24:34.157-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:34.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:24:34.157-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:24:34.157-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:24:34.157-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:34.157-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:34.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:24:34.157-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:34.157415155 -0500 CDT
DcpNozzle 2017-04-25T13:24:34.158-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:24:34.158-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:34.15769207 -0500 CDT
CapiNozzle 2017-04-25T13:24:34.158-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:34.158-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:34.158-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is not running, no need to resetConn
CapiNozzle 2017-04-25T13:24:34.158-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:24:34.158-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:24:34.158-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:24:34.159-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:34.159-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:24:34.159-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:34.159-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:24:34.160-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:24:34.160-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:34.160079268 -0500 CDT
DcpNozzle 2017-04-25T13:24:34.160-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:24:34.160-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:34.160-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:24:34.161-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:24:34.161-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:34.161114878 -0500 CDT
DcpNozzle 2017-04-25T13:24:34.161-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:24:34.161-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:34.161-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:24:34 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:24:34 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:24:34 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:24:34.176-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:24:34.234-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:24:34.234-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:24:34.234-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:34.234-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:24:34.287-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:34.287-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:24:34.288-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:24:34.367-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:34.367-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:24:34.367-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:34.368-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:24:34.368-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:34.368-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:24:34.368-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:24:34.368-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:24:34.368-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
GenericSupervisor 2017-04-25T13:24:34.371-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:24:34.371-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
GenericSupervisor 2017-04-25T13:24:34.371-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:34.371-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:34.371-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:34.371-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:34.371-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:24:34.371-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
CheckpointManager 2017-04-25T13:24:34.371-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
CapiUtils 2017-04-25T13:24:34.411-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:34.411-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:34.411-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:24:34.413-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
StatisticsManager 2017-04-25T13:24:34.414-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:24:34.414-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
XDCRFactory 2017-04-25T13:24:34.414-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:24:34.414-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770 is constructed, start it
GenericPipeline 2017-04-25T13:24:34.416-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[103 122 57 92 91 55 72 115 27 56 118 31 40 102 20 135 125 127 136 33 152 65 8 52 54 154 160 70 119 151 167 96 112 140 98 107 61 121 25 164 6 81 84 137 142 159 169 68 104 147 100 11 23 143 12 73 88 109 26 161 34 53 95 156 69 29 71 32 41 64 83 165 15 21 63 130 170 3 30 38 74 144 145 16 5 10 18 24 117 129 37 166 9 46 99 131 168 80 133 128 138 148 28 48 105 124 141 19 1 47 126 0 13 42 114 49 66 75 86 139 36 60 89 106 158 2 76 108 123 17 110 146 44 62 157 4 149 163 45 79 85 97 111 22 43 58 67 101 113 153 82 90 116 150 39 50 59 162 14 93 134 35 77 51 94 132 78 120 155 87 7] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[220 256 277 282 286 332 185 259 288 253 258 271 285 306 205 239 247 312 323 336 174 186 292 299 238 252 263 295 303 183 294 192 198 210 234 301 209 226 321 333 173 240 300 317 193 206 236 274 176 248 283 335 214 228 237 244 273 188 232 272 293 304 204 211 260 280 257 311 187 194 246 262 269 291 329 216 199 207 235 339 243 254 284 177 213 223 233 268 298 322 190 328 245 215 275 318 231 297 334 181 203 242 249 325 171 172 208 290 327 227 196 217 224 302 310 250 255 276 330 200 230 251 279 229 337 178 180 225 324 340 182 241 309 184 289 320 221 201 270 281 308 212 316 319 261 189 191 222 265 278 296 307 287 314 315 218 313 179 326 197 219 305 202 266 267 331 195 175 264 338]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[125 127 136 33 152 65 8 52 54 154 160 70 119 151 167 96 112 140 98 107 61 121 25 164 6 81 84 137 142 159 169 68 104 147 100 11 23 143 12 73 88 109 26 161 34 53 95 156 69 29 71 32 41 64 83 165 15 21 63 130 170 3 30 38 74 144 145 16 5 10 18 24 117 129 37 166 9 46 99 131 168 80 133 128 138 148 28 48 105 124 141 19 1 47 126 0 13 42 114 49 66 75 86 139 36 60 89 106 158 2 76 108 123 17 110 146 44 62 157 4 149 163 45 79 85 97 111 22 43 58 67 101 113 153 82 90 116 150 39 50 59 162 14 93 134 35 77 51 94 132 78 120 155 87 7 103 122 57 92 91 55 72 115 27 56 118 31 40 102 20 135] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[174 186 292 299 238 252 263 295 303 183 294 192 198 210 234 301 209 226 321 333 173 240 300 317 193 206 236 274 176 248 283 335 214 228 237 244 273 188 232 272 293 304 204 211 260 280 257 311 329 187 194 246 262 269 291 216 199 207 235 339 243 254 284 268 177 213 223 233 298 322 190 328 245 318 215 275 297 231 334 181 203 242 249 325 171 172 208 290 327 227 224 196 217 302 310 250 255 276 330 279 200 230 251 229 337 225 178 180 324 340 182 241 309 184 289 320 221 201 270 281 308 319 212 316 261 265 189 191 222 278 296 307 287 314 315 218 313 179 326 197 219 305 202 266 267 331 195 175 264 338 220 256 277 282 286 332 185 259 288 306 253 258 271 285 205 239 247 312 323 336]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 replication_type:capi filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 checkpoint_interval:1800 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:24:34.417-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770...
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:34.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:34.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:24:34.435-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:34.435-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:24:34.435-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:24:34.435-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:24:34.435-05:00 [INFO] Checkpointing starts in 1802.029 sec
PipelineRuntimeCtx 2017-04-25T13:24:34.435-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:24:34.435-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:24:34.435-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:24:34.435-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:24:34.437-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:24:34.437-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:24:34.437-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:24:34.437-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:24:34.443-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:24:34.474-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:24:34.479-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:34.479-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:34.479-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:24:34.502-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:24:34.508-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:24:34.508-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:24:34.508-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:24:34.511-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:24:34.511-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:24:34.530-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:24:34.575-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:24:34.576-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:34.576-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:24:34.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:24:34.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:24:34.576-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:24:34.616-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:24:34.616-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:24:34.616-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:24:34.638-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:24:34.638-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:24:34.661-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:24:34.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:24:34.736-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:34.736-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:24:34.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:24:34.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:24:34.736-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:24:34.736-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770 is started----------
PipelineManager 2017-04-25T13:24:34.736-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:24:34.736-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:24:34.736-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:24:35.348-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:24:35.348-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:24:35.348-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:24:35.348-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:24:35 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:24:35.440-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:24:35.447-05:00 [INFO] Send batch count=102
2017/04/25 13:24:35 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 301 successful
StatisticsManager 2017-04-25T13:24:35.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:24:35.513-05:00 [INFO] Send batch count=56
2017/04/25 13:24:35 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 4 successful
CapiNozzle 2017-04-25T13:24:35.544-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:35.544-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:35.547-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:35.548-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:35.561-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:24:35 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 306 successful
CapiNozzle 2017-04-25T13:24:35.569-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
2017/04/25 13:24:35 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 221 successful
CapiNozzle 2017-04-25T13:24:35.760-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:35.782-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:24:35 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 19 successful
CapiNozzle 2017-04-25T13:24:35.856-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
2017/04/25 13:24:35 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 325 successful
CapiNozzle 2017-04-25T13:24:35.880-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
2017/04/25 13:24:35 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:24:35 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 156 successful
CapiNozzle 2017-04-25T13:24:36.229-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:36.253-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:24:36.262-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
2017/04/25 13:24:36 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 263 successful
CapiNozzle 2017-04-25T13:24:36.297-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
2017/04/25 13:24:36 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 83 successful
CapiNozzle 2017-04-25T13:24:36.421-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:36.421-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:24:36.421-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:24:36.421-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:24:36.421-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:24:36.431-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:24:36.431-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:24:36.437-05:00 [INFO] err_list=[{"time":"2017-04-25T13:24:36.437476857-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:24:36.437-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:24:36.455-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:24:36.455-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770 {"Errors": "[{\"time\":\"2017-04-25T13:24:36.437476857-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

2017/04/25 13:24:36 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 283 successful
CapiNozzle 2017-04-25T13:24:36.497-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:36.497-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:24:36.497-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:24:36.497-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:36.497-05:00 [INFO] Send batch count=500
CheckpointManager 2017-04-25T13:24:36.501-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
CheckpointManager 2017-04-25T13:24:36.501-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:24:36 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 92 successful
CapiNozzle 2017-04-25T13:24:36.533-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:36.533-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:36.549-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:24:36 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:24:36 UPR_STREAMREQ for vb 298 successful
DcpNozzle 2017-04-25T13:24:36.591-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:24:36.592-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:36.592-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:36.634-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
DcpNozzle 2017-04-25T13:24:36.656-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:24:36.905-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:36.920-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:24:37.097-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:24:37.111-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:24:37.292-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:37.325-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:24:37.473-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:24:37.490-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:24:37.776-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:37.776-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:24:37.776-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:24:37.776-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:37.776-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:37.888-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:37.897-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:37.912-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:24:38.052-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:24:38.052-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:24:38.052-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:24:38.052-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:38.052-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:38.141-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:38.141-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:38.156-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:24:38.211-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:38.228-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:24:38.514-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:38.526-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:24:38.613-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:38.624-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:24:38.939-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:38.961-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:24:39.032-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:39.032-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:24:39.032-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:24:39.032-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:39.032-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:39.118-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:39.118-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:39.140-05:00 [INFO] Retrying update docs for vb 107 for the 1th time
CapiNozzle 2017-04-25T13:24:39.353-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:39.353-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:24:39.353-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:24:39.353-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:39.353-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:39.409-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:39.410-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:39.440-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:24:39.539-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:24:39.568-05:00 [INFO] Retrying update docs for vb 107 for the 2th time
PipelineManager 2017-04-25T13:24:39.651-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:36.437476857-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:24:40.451-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:24:40.460-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:40.483-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:24:40.505-05:00 [INFO] Retrying update docs for vb 107 for the 3th time
CapiNozzle 2017-04-25T13:24:40.849-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:40.880-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:24:40.980-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:24:40.980-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 107 after 3 retries
CapiNozzle 2017-04-25T13:24:40.980-05:00 [ERROR] Raise error condition batch update docs failed for vb 107 after 3 retries
GenericSupervisor 2017-04-25T13:24:40.980-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:40.981-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:41.068-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:41.068-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:41.082-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:24:41.202-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:41.202-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:24:41.202-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:24:41.202-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:41.202-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:41.278-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:41.278-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:41.292-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:24:42.069-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:42.085-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:24:42.385-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:24:42.416-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:24:42.565-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:42.577-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:24:42.812-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:24:42.824-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:24:42.994-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:24:42.994-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:24:42.994-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:24:42.994-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:42.995-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:43.046-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:43.046-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:43.060-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:24:43.504-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:24:43.504-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:24:43.504-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:24:43.504-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:43.504-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:43.572-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:43.574-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:43.587-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:24:43.749-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:43.767-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:24:44.040-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:24:44.051-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:24:44.222-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:44.233-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
GenericSupervisor 2017-04-25T13:24:44.436-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:24:44.436-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:24:44.436-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:24:44.564-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:24:44.579-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:24:44.684-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:24:44.684-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:24:44.684-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:24:44.684-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:44.684-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:24:44.686-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:24:44.762-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:44.763-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:44.773-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:24:44.934-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:24:44.934-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:24:44.934-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:24:44.934-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:44.934-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:45.057-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:45.057-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:45.068-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:24:45.097-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:24:45.108-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:24:45.449-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:45.459-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:24:45.657-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:24:45.668-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:24:46.055-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:46.066-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:24:46.367-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:24:46.367-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:24:46.367-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:24:46.367-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:46.367-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:24:46.437-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:24:46.437-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:24:46.437-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:24:46.437-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
PipelineRuntimeCtx 2017-04-25T13:24:46.437-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:24:46.437-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:46.437-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:24:46.438-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:46.437989638 -0500 CDT
GenericSupervisor 2017-04-25T13:24:46.438-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:46.438-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:24:46.438-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:24:46.438-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:24:46.438-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:24:46.438-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:24:46.438-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:24:46.438-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:24:46.438-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:24:46.438-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-368182770 has been stopped
GenericPipeline 2017-04-25T13:24:46.438-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-368182770 is stopped
PipelineManager 2017-04-25T13:24:46.438-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:24:46.438-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:36.437476857-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:24:46.438-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:24:46.438-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
GenericPipeline 2017-04-25T13:24:46.438-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
GenericPipeline 2017-04-25T13:24:46.438-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
GenericPipeline 2017-04-25T13:24:46.439-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-368182770
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:24:46.439-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:24:46.439-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:46.439-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:24:46.439-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:24:46.439-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:24:46.439-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:24:46.439-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:46.440-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:24:46.440-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:24:46.440-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:24:46.440-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:24:46.440-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:24:46.440-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:24:46.440-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:46.440005318 -0500 CDT
DcpNozzle 2017-04-25T13:24:46.440-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:24:46.440-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:46.440304753 -0500 CDT
CapiNozzle 2017-04-25T13:24:46.441-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:24:46.441-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:24:46.441-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:24:46.441-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:46.441-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:24:46.442-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:24:46.442-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:46.442-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:24:46 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:24:46.447-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:24:46.447-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:46.447777393 -0500 CDT
DcpNozzle 2017-04-25T13:24:46.447-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:24:46.448-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:24:46 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:24:46 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:24:46 Stream Closed for vb 174 StreamEnd simulated
DcpNozzle 2017-04-25T13:24:46.448-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:24:46.448002825 -0500 CDT
GenericPipeline 2017-04-25T13:24:46.448-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:46.448-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:24:46.448-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:24:46.448-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:46.456-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:24:46 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:24:46 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:24:46 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:24:46 Stream Closed for vb 3 StreamEnd simulated
CapiNozzle 2017-04-25T13:24:46.504-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:24:46.504-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:24:46.504-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:24:46.504-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:24:46.576-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:46.576-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:24:46.576-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:24:46.623-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:24:46.623-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:24:46.623-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:46.623-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:24:46.623-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:24:46.623-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:24:46.624-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:24:46.624-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:24:46.625-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
GenericSupervisor 2017-04-25T13:24:46.627-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:24:46.627-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
GenericSupervisor 2017-04-25T13:24:46.627-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:46.627-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:46.627-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:46.627-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:24:46.627-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:24:46.627-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
CheckpointManager 2017-04-25T13:24:46.627-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
CapiUtils 2017-04-25T13:24:46.659-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:46.659-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:46.659-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:24:46.660-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
StatisticsManager 2017-04-25T13:24:46.660-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:24:46.661-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
XDCRFactory 2017-04-25T13:24:46.661-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:24:46.661-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204 is constructed, start it
GenericPipeline 2017-04-25T13:24:46.662-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[96 130 137 12 46 77 94 108 115 120 21 34 43 134 156 168 7 54 104 155 50 111 149 95 167 157 158 1 49 64 79 30 44 58 65 126 141 48 24 89 117 4 85 131 8 145 152 92 98 17 20 57 76 100 119 112 19 38 99 31 53 55 118 169 2 82 88 124 170 52 84 132 139 23 33 90 106 74 147 160 9 37 42 86 10 11 66 16 97 163 5 40 41 162 6 35 123 127 22 60 114 161 105 166 83 142 148 150 164 36 102 116 125 73 61 39 144 25 29 72 128 3 87 93 107 47 151 13 109 62 71 80 51 138 14 159 67 59 113 129 26 32 56 146 18 154 27 122 103 15 70 110 143 121 165 28 45 63 78 91 101 133 0 136 140 153 81 135 68 75 69] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[302 189 224 259 321 221 193 237 290 256 222 257 307 207 234 303 330 184 206 213 235 288 297 294 210 242 243 177 194 204 295 292 191 198 246 272 304 260 179 218 282 339 200 202 251 286 338 226 261 311 176 231 247 266 283 327 180 183 205 214 310 270 284 305 249 262 185 203 248 285 319 182 252 267 298 301 254 278 188 192 245 187 300 318 326 336 197 212 328 225 255 296 172 276 317 199 173 208 209 250 299 314 241 244 332 201 230 271 312 171 263 268 264 217 329 335 340 211 274 325 306 334 227 322 174 273 308 228 323 223 196 233 236 265 178 275 279 337 190 309 258 289 331 181 253 239 293 186 195 238 313 215 216 291 315 316 320 324 277 220 280 219 232 287 229 333 175 269 240 281]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[156 168 7 54 104 155 50 111 149 95 167 157 158 1 49 64 79 30 44 58 65 126 141 48 24 89 117 4 85 131 8 145 152 92 98 17 20 57 76 100 119 112 19 38 99 31 53 55 118 169 2 82 88 124 170 52 84 132 139 23 33 90 106 74 147 160 9 37 42 86 10 11 66 16 97 163 5 40 41 162 6 35 123 127 22 60 114 161 105 166 83 142 148 150 164 36 102 116 125 73 61 39 144 25 29 72 128 3 87 93 107 47 151 13 109 62 71 80 51 138 14 159 67 59 113 129 26 32 56 146 18 154 27 122 103 15 70 110 143 121 165 28 45 63 78 91 101 133 0 136 140 153 81 135 68 75 69 96 130 137 12 46 77 94 108 115 120 21 34 43 134] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[237 290 256 222 257 307 207 234 303 330 184 206 213 235 288 297 294 210 242 243 177 194 204 295 292 191 198 246 272 304 260 179 218 282 339 200 202 251 286 338 226 261 311 176 231 247 266 283 327 180 183 205 214 310 270 284 305 249 262 185 203 248 285 319 182 252 267 298 301 254 278 188 192 245 187 300 318 326 336 197 212 328 225 255 296 172 276 317 199 173 208 209 250 299 314 241 244 332 201 230 271 312 171 263 268 264 217 329 335 340 211 274 325 306 334 227 322 174 273 308 228 323 223 196 233 236 265 178 275 279 337 190 309 258 289 331 181 253 239 293 186 195 238 313 215 216 291 315 316 320 324 277 220 280 219 232 287 229 333 175 269 240 281 302 189 224 259 321 221 193]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 target_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression: active:true optimistic_replication_threshold:256 source_nozzle_per_node:2 replication_type:capi failure_restart_interval:10]
CheckpointManager 2017-04-25T13:24:46.662-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204...
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:24:46.675-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
StatisticsManager 2017-04-25T13:24:46.675-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:24:46.688-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:24:46.688-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:24:46.688-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:24:46.689-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:24:46.689-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:24:46.689-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:24:46.689-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:24:46.689-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:24:46.689-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:24:46.689-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:24:46.689-05:00 [INFO] Checkpointing starts in 1801.093 sec
PipelineRuntimeCtx 2017-04-25T13:24:46.689-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:24:46.689-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:24:46.690-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:24:46.691-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:24:46.693-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:24:46.703-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:24:46.731-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:24:46.732-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:24:46.732-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:24:46.743-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:24:46.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:24:46.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:24:46.762-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:24:46.780-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:24:46.781-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:46.781-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:24:46.781-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:24:46.781-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:24:46.781-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:24:46.818-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:24:46.818-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:24:46.818-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:24:46.848-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:24:46.848-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:24:46.952-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:24:46.973-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:24:46.973-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:24:46.973-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:24:46.973-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:24:46.973-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:24:46.973-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:24:46.973-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204 is started----------
PipelineManager 2017-04-25T13:24:46.973-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:24:46.973-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:24:46.973-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:24:47.791-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:24:48.805-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:24:49.819-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:24:50.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:24:51.776-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:24:52.583-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:24:52.583-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:24:52.583-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:24:52.583-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:24:52 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:24:52.680-05:00 [INFO] Send batch count=500
2017/04/25 13:24:52 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:24:52.718-05:00 [INFO] Send batch count=48
CapiNozzle 2017-04-25T13:24:52.735-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:52.735-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:52.771-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:24:52.776-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:24:52 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 5 successful
CapiNozzle 2017-04-25T13:24:52.777-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:24:52 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 9 successful
StatisticsManager 2017-04-25T13:24:52.805-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:24:52 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:24:52.825-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:24:52 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 312 successful
CapiNozzle 2017-04-25T13:24:52.894-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:52.905-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
2017/04/25 13:24:52 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:24:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 33 successful
CapiNozzle 2017-04-25T13:24:53.166-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:53.177-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:53.181-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:24:53.206-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
2017/04/25 13:24:53 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 152 successful
CapiNozzle 2017-04-25T13:24:53.309-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:24:53.309-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:24:53.309-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:24:53.309-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries]
ReplicationManager 2017-04-25T13:24:53.309-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries]
2017/04/25 13:24:53 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 262 successful
PipelineManager 2017-04-25T13:24:53.451-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:24:53.451-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:24:53.455-05:00 [INFO] err_list=[{"time":"2017-04-25T13:24:53.45599289-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries"}]
PipelineManager 2017-04-25T13:24:53.456-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:24:53 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 167 successful
CapiNozzle 2017-04-25T13:24:53.549-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:53.549-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:53.578-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:24:53 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 73 successful
CapiNozzle 2017-04-25T13:24:53.660-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:24:53 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 275 successful
CapiNozzle 2017-04-25T13:24:53.694-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:24:53.710-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:24:53.710-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204 {"Errors": "[{\"time\":\"2017-04-25T13:24:53.45599289-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 1575, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 1575, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries]", "Status": "Pending"}

2017/04/25 13:24:53 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 193 successful
CheckpointManager 2017-04-25T13:24:53.916-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
CheckpointManager 2017-04-25T13:24:53.916-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:24:53 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:24:53 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:24:54.100-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:24:54.100-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:24:54.187-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:54.199-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:24:54.344-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:24:54.344-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:24:54.344-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:24:54.344-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:54.344-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:54.429-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:54.429-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:54.450-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:24:54.654-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
ReplicationManager 2017-04-25T13:24:54.661-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:24:54.676-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
PipelineManager 2017-04-25T13:24:54.808-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:53.45599289-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries]}
]
CapiNozzle 2017-04-25T13:24:54.863-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:54.880-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:24:55.074-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:24:55.074-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:24:55.074-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:24:55.074-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:55.074-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:55.185-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:55.185-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:55.206-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:24:55.397-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:55.427-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:24:55.744-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:55.768-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:24:55.946-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:24:55.946-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:24:55.946-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:24:55.946-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:55.946-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:56.027-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:56.027-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:56.045-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:24:56.110-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:56.122-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:24:56.456-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:24:56.472-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:24:56.666-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:24:56.666-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:24:56.666-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:24:56.666-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:56.666-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:24:56.695-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:24:56.695-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:24:56.695-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:24:56.759-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:56.759-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:56.786-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:24:56.910-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:24:56.936-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
ToplogyChangeDetector 2017-04-25T13:24:56.960-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:24:57.336-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:24:57.360-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:24:57.470-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:24:57.470-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:24:57.470-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:24:57.470-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:57.470-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:57.621-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:57.621-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:57.642-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:24:57.968-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:24:57.981-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:24:58.662-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:24:58.685-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:24:59.108-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:24:59.108-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:24:59.108-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:24:59.108-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:24:59.108-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:24:59.187-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:24:59.187-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:24:59.201-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:25:00.751-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:25:00.753-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:25:00.775-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:25:00.794-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:25:01.117-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:25:01.142-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:25:01.375-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:25:01.375-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:25:01.375-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:25:01.375-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:01.375-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:01.480-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:01.480-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:01.491-05:00 [INFO] Retrying update docs for vb 6 for the 1th time
CapiNozzle 2017-04-25T13:25:01.587-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:25:01.587-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:25:01.587-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:25:01.587-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:01.587-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:01.687-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:01.687-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:01.698-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:25:01.958-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:25:01.969-05:00 [INFO] Retrying update docs for vb 6 for the 2th time
CapiNozzle 2017-04-25T13:25:02.331-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:02.342-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:25:02.549-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:25:02.560-05:00 [INFO] Retrying update docs for vb 6 for the 3th time
PipelineManager 2017-04-25T13:25:03.456-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries 
PipelineManager 2017-04-25T13:25:03.456-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:25:03.456-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:03.456-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
PipelineRuntimeCtx 2017-04-25T13:25:03.456-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:25:03.456-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:03.456-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:25:03.456-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:03.456621467 -0500 CDT
GenericSupervisor 2017-04-25T13:25:03.456-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:03.456-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:25:03.456-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:03.456-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:25:03.456-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:03.456-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:25:03.456-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:25:03.456-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:25:03.456-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:25:03.456-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-624030204 has been stopped
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-624030204 is stopped
PipelineManager 2017-04-25T13:25:03.457-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
PipelineManager 2017-04-25T13:25:03.457-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:24:53.45599289-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 304 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
PipelineManager 2017-04-25T13:25:03.457-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-624030204
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:25:03.457-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:25:03.457-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:03.457-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:03.457-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:25:03.457-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:03.458-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:25:03.458-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:25:03.458-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:25:03.458-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:25:03.458-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:25:03.458-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:25:03.458-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:03.458297286 -0500 CDT
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:25:03.458-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:03.457991134 -0500 CDT
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:25:03.458-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:03.458-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:25:03.459-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:25:03.459-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:03.459-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:25:03.459-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:03.459-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:25:03.459-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:25:03.460-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:03.460-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:25:03.460-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:03.460-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:25:03.460-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:03.460-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:03.460768184 -0500 CDT
DcpNozzle 2017-04-25T13:25:03.460-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:25:03.460-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:03.460-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:25:03.462-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:03.462-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:03.462563441 -0500 CDT
DcpNozzle 2017-04-25T13:25:03.462-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:25:03.462-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:03.462-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:25:03 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:25:03 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:25:03 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:25:03 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:25:03.577-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:03.578-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:25:03.578-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:25:03.657-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:03.658-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:25:03.658-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:03.658-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:25:03.658-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:03.658-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:25:03.658-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:25:03.658-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:25:03.659-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
GenericSupervisor 2017-04-25T13:25:03.660-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:25:03.661-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
GenericSupervisor 2017-04-25T13:25:03.661-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:03.661-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:03.661-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:03.661-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:03.661-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:03.661-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
CheckpointManager 2017-04-25T13:25:03.661-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
CapiUtils 2017-04-25T13:25:03.717-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:03.717-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:03.717-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:25:03.718-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
StatisticsManager 2017-04-25T13:25:03.718-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:25:03.718-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
XDCRFactory 2017-04-25T13:25:03.718-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:25:03.719-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904 is constructed, start it
GenericPipeline 2017-04-25T13:25:03.720-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[166 5 10 61 142 114 117 21 41 103 155 121 19 73 134 30 132 165 0 15 39 162 58 85 91 133 170 59 86 144 55 161 168 33 52 147 143 23 102 135 43 125 25 45 163 3 28 92 100 14 101 4 54 115 119 20 75 95 13 18 47 146 48 94 56 76 64 108 8 97 140 24 36 44 50 152 16 150 158 37 159 160 40 74 82 88 107 122 106 139 26 57 62 66 164 34 46 129 131 17 113 137 38 128 130 154 72 104 109 123 112 78 80 83 90 69 118 2 110 87 120 145 157 6 53 71 93 49 124 141 42 67 81 98 65 79 89 96 11 31 35 63 153 167 32 99 111 156 68 127 148 51 77 136 70 149 9 12 22 105 7 60 1 126 151 27 29 84 116 138 169] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[175 318 232 189 262 269 315 207 310 304 305 329 285 195 259 248 299 219 176 239 312 206 283 317 226 326 288 307 309 191 211 214 220 270 332 255 197 243 336 192 289 178 185 251 273 249 250 186 225 227 272 276 278 201 182 313 319 234 244 267 271 188 194 287 324 252 294 320 173 265 204 257 215 264 286 308 228 303 263 202 221 213 297 327 311 181 238 261 292 293 196 216 230 280 205 209 223 236 174 203 241 190 218 180 229 296 274 187 323 302 322 333 177 183 300 338 193 231 260 266 282 339 334 295 330 172 242 298 306 268 208 224 291 325 275 277 281 316 184 337 247 179 222 245 328 199 235 253 284 301 217 233 290 314 331 258 321 212 240 254 200 198 210 237 335 340 171 256 279 246]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[221 213 297 327 181 238 261 311 216 230 280 292 293 196 174 203 205 209 223 236 241 190 218 180 296 229 274 187 323 177 183 302 322 333 300 338 193 231 260 266 339 282 334 295 330 172 242 298 306 268 208 224 291 325 184 275 277 281 316 337 179 222 247 245 328 235 253 284 199 217 233 301 290 314 331 212 240 258 321 200 254 198 210 237 335 171 256 279 340 246 175 318 189 232 262 269 315 207 310 304 305 329 195 259 285 219 248 299 176 239 312 206 226 283 317 288 307 309 326 191 211 214 220 255 270 332 197 243 192 289 336 178 185 249 250 251 273 186 225 227 272 276 201 278 182 313 319 188 194 234 244 267 271 324 287 252 294 173 265 320 204 257 215 264 308 286 228 303 202 263] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[74 40 88 107 122 82 57 62 66 106 139 26 164 46 34 131 129 113 137 17 38 104 109 123 128 130 154 72 80 83 90 112 78 118 69 110 2 120 145 157 87 53 71 93 6 124 49 67 81 98 141 42 31 35 63 65 79 89 96 11 167 153 99 111 156 32 127 148 68 77 51 136 149 70 12 22 105 9 60 7 126 151 1 29 84 116 27 138 169 10 61 142 166 5 117 114 41 103 21 155 121 19 134 73 132 165 30 15 39 162 0 85 91 133 58 170 86 144 59 161 55 168 52 147 33 143 102 135 23 125 43 45 25 28 92 100 163 3 101 14 54 115 119 4 75 95 20 18 47 146 13 94 48 76 56 108 64 8 36 44 50 97 140 24 152 150 158 16 159 160 37]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info replication_type:capi checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 stats_interval:1000 filter_expression: active:true]
CheckpointManager 2017-04-25T13:25:03.720-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904...
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:03.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:25:03.738-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:25:03.738-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:25:03.738-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:03.738-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:25:03.738-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:25:03.738-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:25:03.738-05:00 [INFO] Checkpointing starts in 1801.132 sec
PipelineRuntimeCtx 2017-04-25T13:25:03.738-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:25:03.738-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:25:03.738-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:25:03.738-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:25:03.741-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:25:03.741-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:25:03.743-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:25:03.767-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:25:03.767-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:25:03.767-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:25:03.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:25:03.784-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:03.784-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:03.784-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:25:03.785-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:03.785-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:25:03.788-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:25:03.793-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:25:03.850-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:25:03.850-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:25:03.850-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:03.850-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:25:03.851-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:25:03.851-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:25:03.851-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:25:03.866-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:25:03.866-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:25:03.866-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:25:03.883-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:03.883-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:04.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:25:04.001-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:04.001-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:25:04.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:25:04.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:25:04.002-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:25:04.002-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904 is started----------
PipelineManager 2017-04-25T13:25:04.002-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:25:04.002-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:25:04.002-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:25:04.161-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:25:04.162-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:25:04.162-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:25:04.162-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:25:04 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:25:04.252-05:00 [INFO] Send batch count=120
2017/04/25 13:25:04 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:25:04.300-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:25:04.322-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:04.322-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:25:04 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:25:04.331-05:00 [INFO] Send batch count=169
CapiNozzle 2017-04-25T13:25:04.366-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:25:04 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:25:04.429-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:04.429-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:04.444-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:25:04 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 328 successful
CapiNozzle 2017-04-25T13:25:04.744-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:04.772-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:25:04 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 35 successful
StatisticsManager 2017-04-25T13:25:04.858-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:25:04 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 53 successful
CapiNozzle 2017-04-25T13:25:04.916-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:25:04 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 251 successful
CapiNozzle 2017-04-25T13:25:04.934-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:25:04 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:25:04 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 88 successful
CapiNozzle 2017-04-25T13:25:05.274-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:05.321-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:25:05 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 293 successful
CheckpointManager 2017-04-25T13:25:05.443-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
CheckpointManager 2017-04-25T13:25:05.443-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:25:05 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:25:05 UPR_STREAMREQ for vb 297 successful
DcpNozzle 2017-04-25T13:25:05.560-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:25:05.594-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
DcpNozzle 2017-04-25T13:25:05.603-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:25:05.620-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
StatisticsManager 2017-04-25T13:25:05.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:25:05.847-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:05.847-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:25:05.847-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:25:05.847-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:25:05.847-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:25:05.861-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:25:05.861-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:25:05.863-05:00 [INFO] err_list=[{"time":"2017-04-25T13:25:05.863796206-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:25:05.863-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:25:05.923-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:05.925-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:05.943-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:25:05.999-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:05.999-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:25:05.999-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:25:05.999-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:06.000-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:06.073-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:06.074-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:06.093-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:25:06.539-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:06.553-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:25:06.675-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:06.689-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
StatisticsManager 2017-04-25T13:25:06.744-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:25:06.745-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904 {"Errors": "[{\"time\":\"2017-04-25T13:25:05.863796206-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 126, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 68734, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 126, "set_filtered": 0, "set_received_from_dcp": 68734, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:25:06.899-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:06.923-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:25:07.036-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:07.059-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:25:07.274-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:07.274-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:25:07.274-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:25:07.274-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:07.274-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:07.334-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:07.334-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:07.375-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:25:07.416-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:07.416-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:25:07.416-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:25:07.416-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:07.416-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:07.473-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:07.473-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:07.489-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:25:07.631-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:07.644-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:25:07.874-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:25:07.888-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:25:08.012-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:08.030-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:25:08.191-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:25:08.207-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:25:08.344-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:08.344-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:25:08.344-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:25:08.344-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:08.344-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:08.462-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:08.462-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:08.478-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:25:08.602-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:25:08.602-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:25:08.602-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:25:08.602-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:08.602-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:08.689-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:08.689-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:08.703-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:25:08.803-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:08.820-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:25:09.155-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:09.173-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:25:09.288-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:09.306-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:25:09.541-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:09.560-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
PipelineManager 2017-04-25T13:25:09.736-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:05.863796206-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:25:09.863-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:09.863-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:25:09.863-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:25:09.863-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:09.863-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:09.960-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:09.960-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:09.987-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:25:10.190-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:10.190-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:25:10.190-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:25:10.190-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:10.190-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:10.297-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:10.297-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:10.313-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:25:10.379-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:10.400-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:25:10.666-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:10.682-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:25:10.918-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:10.945-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:25:11.351-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:11.375-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:25:11.585-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:11.585-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:25:11.585-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:25:11.585-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:11.585-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:11.655-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:11.655-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:11.675-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:25:12.031-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:12.031-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:25:12.031-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:25:12.031-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:12.031-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:12.125-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:12.126-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:12.136-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:25:12.319-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:25:12.330-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:25:12.492-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:12.503-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:25:12.646-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:25:12.656-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:25:12.816-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:12.827-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:25:13.004-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:25:13.004-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:25:13.004-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:25:13.004-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:13.004-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:13.063-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:13.063-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:13.073-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:25:13.286-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:13.286-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:25:13.286-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:25:13.286-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:13.287-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:13.441-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:13.441-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:13.451-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
GenericSupervisor 2017-04-25T13:25:13.738-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:25:13.739-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:25:13.739-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:25:13.862-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:25:13.873-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
ToplogyChangeDetector 2017-04-25T13:25:13.884-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:25:14.648-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:25:14.659-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:25:14.880-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:25:14.891-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:25:15.490-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:25:15.501-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:25:15.677-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:25:15.677-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:25:15.677-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:25:15.677-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:15.677-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:15.752-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:15.752-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:15.762-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
PipelineManager 2017-04-25T13:25:15.864-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:25:15.864-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:25:15.864-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:15.864-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
PipelineRuntimeCtx 2017-04-25T13:25:15.864-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:25:15.864-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:25:15.864-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:15.864-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:25:15.864-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:15.864362148 -0500 CDT
GenericSupervisor 2017-04-25T13:25:15.864-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:15.864-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:25:15.864-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:15.864-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:25:15.864-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:15.864-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:25:15.864-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:25:15.864-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:25:15.864-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-658524904 has been stopped
GenericPipeline 2017-04-25T13:25:15.864-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-658524904 is stopped
PipelineManager 2017-04-25T13:25:15.864-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:25:15.864-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:05.863796206-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:25:15.865-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:15.865-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
GenericPipeline 2017-04-25T13:25:15.865-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
GenericPipeline 2017-04-25T13:25:15.865-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
GenericPipeline 2017-04-25T13:25:15.865-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-658524904
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:15.865-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:15.866-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:25:15.866-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:15.866-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:25:15.866-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:15.866-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:25:15.866-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:15.866-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:25:15.866-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:15.866-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:15.866-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:25:15.867-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:15.866457221 -0500 CDT
DcpNozzle 2017-04-25T13:25:15.867-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:25:15.867-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:15.866749312 -0500 CDT
CapiNozzle 2017-04-25T13:25:15.868-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:25:15.868-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:15.868-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:25:15.868-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:15.868-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:25:15.868-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:25:15.868-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.868-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:15.868-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.868-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:15.868-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.868-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:15.869-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:15.869-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:25:15.869-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:15.869-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:25:15.873-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:15.873-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:15.873-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:15.873873875 -0500 CDT
DcpNozzle 2017-04-25T13:25:15.873-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:15.873891144 -0500 CDT
DcpNozzle 2017-04-25T13:25:15.873-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:25:15.873-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:25:15.874-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:15.874-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:25:15.874-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:15.874-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:25:15 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:25:15 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:25:15 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:25:15 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:25:15 Stream Closed for vb 2 StreamEnd simulated
XDCRFactory 2017-04-25T13:25:15.990-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:15.990-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:25:15.990-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:25:16.033-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:16.033-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:25:16.034-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:16.034-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:25:16.034-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:16.034-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:25:16.034-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:25:16.034-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:25:16.035-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
GenericSupervisor 2017-04-25T13:25:16.036-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:25:16.036-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
GenericSupervisor 2017-04-25T13:25:16.036-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:16.036-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:16.036-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:16.036-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:16.037-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:16.037-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
CheckpointManager 2017-04-25T13:25:16.037-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
CapiUtils 2017-04-25T13:25:16.065-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:16.065-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:16.065-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:25:16.066-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
StatisticsManager 2017-04-25T13:25:16.066-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:25:16.066-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
XDCRFactory 2017-04-25T13:25:16.067-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:25:16.067-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063 is constructed, start it
GenericPipeline 2017-04-25T13:25:16.068-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[167 17 21 44 60 78 101 142 5 105 106 76 90 96 127 23 141 144 53 146 163 36 37 45 24 63 139 164 18 20 74 155 7 13 39 136 10 89 147 107 157 14 30 93 99 102 11 64 77 150 153 3 32 92 117 133 54 68 71 91 124 41 61 12 168 70 83 22 120 129 149 119 160 65 135 33 62 98 80 118 123 126 128 148 69 97 108 52 109 169 55 67 110 4 16 31 38 46 51 9 43 75 130 166 8 66 15 94 115 29 125 161 138 50 81 113 0 73 156 42 88 100 132 47 111 84 56 72 79 143 152 87 40 57 82 131 151 154 26 59 116 165 35 145 1 49 114 137 158 48 159 2 28 112 162 25 95 122 170 19 134 58 104 6 27 34 103 121 85 86 140] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[247 275 279 300 322 298 314 210 176 180 212 187 233 272 172 205 324 340 186 330 331 333 177 284 305 261 289 323 184 203 207 252 174 240 250 274 232 299 327 263 307 335 312 222 224 317 229 194 218 235 226 264 321 230 251 260 266 280 219 286 241 329 334 199 236 246 282 296 319 182 193 223 265 277 283 310 181 278 287 303 200 256 268 332 339 258 195 234 337 225 255 336 245 309 178 294 316 318 209 185 217 249 281 183 248 304 328 191 231 257 313 269 271 173 308 206 216 288 208 306 192 320 215 239 214 338 244 198 204 238 175 270 290 254 179 237 211 213 221 285 295 196 243 292 228 326 276 190 227 262 291 301 315 311 188 293 220 267 297 201 242 189 273 325 197 202 171 253 259 302]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[127 76 90 96 23 141 144 53 146 163 36 37 45 164 24 63 139 155 18 20 74 136 7 13 39 10 89 147 107 157 99 102 14 30 93 150 153 11 64 77 117 133 3 32 92 91 124 54 68 71 41 61 12 168 70 83 22 120 129 149 119 160 65 135 33 62 98 126 128 148 80 118 123 69 97 108 52 109 169 55 67 110 38 46 51 4 16 31 130 166 9 43 75 8 66 15 94 115 29 125 161 138 50 81 113 0 73 156 132 42 88 100 47 111 84 56 152 72 79 143 87 131 151 154 40 57 82 165 26 59 116 35 145 1 158 49 114 137 48 159 2 28 112 162 25 95 122 170 19 134 58 104 103 121 6 27 34 85 86 140 167 17 21 101 142 44 60 78 5 105 106] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[172 205 324 340 330 331 333 186 305 177 284 289 323 261 184 203 207 174 240 250 252 274 232 299 327 263 307 335 222 224 312 317 194 218 229 235 226 264 251 260 266 280 321 230 286 219 329 334 241 246 282 296 319 199 236 223 182 193 277 265 310 283 278 287 303 181 200 256 268 332 258 339 195 234 337 225 255 245 336 178 294 309 316 318 209 185 217 249 281 183 248 304 328 257 313 191 231 269 271 173 308 206 216 288 208 306 320 192 215 239 214 338 198 204 238 244 290 175 270 179 254 211 213 221 237 285 295 243 292 196 228 326 276 227 262 190 291 301 315 311 188 293 297 220 267 242 201 273 189 197 202 325 171 253 259 302 279 300 322 247 275 314 298 176 180 210 212 187 233 272]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:25:16.068-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063...
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:16.077-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:25:16.077-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:25:16.077-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:25:16.077-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:16.077-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:25:16.077-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:25:16.077-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:25:16.077-05:00 [INFO] Checkpointing starts in 1800.027 sec
PipelineRuntimeCtx 2017-04-25T13:25:16.077-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:25:16.077-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:25:16.078-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:25:16.078-05:00 [INFO] checkpointing rountine started
ToplogyChangeDetector 2017-04-25T13:25:16.087-05:00 [INFO] validateSourceTopology...
PipelineRuntimeCtx 2017-04-25T13:25:16.087-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:25:16.087-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:25:16.089-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:25:16.113-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:25:16.113-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:25:16.113-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:25:16.119-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:16.119-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:16.119-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:25:16.130-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:16.130-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:25:16.140-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:25:16.140-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:25:16.175-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:25:16.175-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:16.175-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:25:16.176-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:25:16.176-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:25:16.176-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:25:16.190-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:25:16.190-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:25:16.193-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:25:16.209-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:16.209-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:25:16.249-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:25:16.333-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:25:16.333-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:16.333-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:25:16.333-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:25:16.333-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:25:16.333-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:25:16.333-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063 is started----------
PipelineManager 2017-04-25T13:25:16.333-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:25:16.333-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:25:16.333-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:25:16.530-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:25:16.531-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:25:16.531-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:25:16.531-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:25:16 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:25:16.633-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:25:16.639-05:00 [ERROR] Received malformed response from tcp connection vb=101. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f103/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:25:16.639-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err Received malformed response from tcp connection vb=101. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f103/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:25:16.653-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:25:16 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:25:16.672-05:00 [INFO] Send batch count=455
CapiNozzle 2017-04-25T13:25:16.675-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:25:16.676-05:00 [INFO] Send batch count=25
CapiNozzle 2017-04-25T13:25:16.703-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:16.703-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:16.724-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:25:16 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:25:16 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 72 successful
StatisticsManager 2017-04-25T13:25:17.140-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=7, changes_left=315501
2017/04/25 13:25:17 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 186 successful
CheckpointManager 2017-04-25T13:25:17.263-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
CheckpointManager 2017-04-25T13:25:17.263-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:25:17 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:25:17 UPR_STREAMREQ for vb 296 successful
DcpNozzle 2017-04-25T13:25:17.378-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:25:17.405-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:17.433-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
DcpNozzle 2017-04-25T13:25:17.437-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:25:17.583-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:17.616-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:17.616-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:25:17.639-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:25:17.687-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:17.687-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:25:17.687-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:25:17.687-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:25:17.687-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:25:17.706-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:25:17.706-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:25:17.709-05:00 [INFO] err_list=[{"time":"2017-04-25T13:25:17.709351454-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:25:17.709-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:25:17.815-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:17.815-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:17.833-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:25:17.960-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:17.980-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
StatisticsManager 2017-04-25T13:25:18.095-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:25:18.095-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063 {"Errors": "[{\"time\":\"2017-04-25T13:25:17.709351454-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315501, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 283, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 7, "docs_received_from_dcp": 19457, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 283, "set_filtered": 0, "set_received_from_dcp": 19459, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:25:18.169-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:25:18.183-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:25:18.322-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:18.338-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:25:18.536-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:25:18.554-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:25:18.757-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:18.757-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:25:18.757-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:25:18.757-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:18.757-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:18.838-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:18.838-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:18.857-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:25:19.078-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:25:19.078-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:25:19.078-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:25:19.078-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:19.078-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:19.176-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:19.176-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:19.196-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:25:19.264-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:19.281-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:25:19.512-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:19.534-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:25:19.662-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:19.677-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:25:20.331-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:20.345-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:25:20.502-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:20.502-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:25:20.502-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:25:20.502-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:20.502-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:20.604-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:20.604-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:20.631-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:25:20.832-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:20.832-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:25:20.832-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:25:20.832-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:20.832-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:20.929-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:20.929-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:20.945-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:25:20.949-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:20.982-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:25:21.422-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:21.435-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:25:21.738-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:21.773-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:25:21.969-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:21.969-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:25:21.969-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:25:21.969-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:21.969-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:22.107-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:22.107-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:22.122-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:25:22.285-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:22.296-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:25:22.547-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:22.590-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:25:22.698-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:22.698-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:25:22.698-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:25:22.698-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:22.698-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:22.784-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:22.784-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:22.820-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:25:23.057-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:23.080-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:25:23.253-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:23.271-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:25:23.465-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:23.465-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:25:23.465-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:25:23.465-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:23.465-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:23.541-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:23.541-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:23.557-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:25:23.617-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:23.633-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:25:24.318-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:24.329-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:25:24.468-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:24.468-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:25:24.468-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:25:24.468-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:24.468-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:24.541-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:24.541-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:24.552-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
PipelineManager 2017-04-25T13:25:24.570-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:17.709351454-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:25:24.864-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:24.874-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:25:25.053-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:25:25.064-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:25:25.436-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:25:25.436-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:25:25.436-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:25:25.436-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:25.436-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:25.469-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 10
CapiNozzle 2017-04-25T13:25:25.520-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:25.520-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:25.530-05:00 [INFO] Retrying update docs for vb 10 for the 1th time
CapiNozzle 2017-04-25T13:25:25.927-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:25:25.938-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
GenericSupervisor 2017-04-25T13:25:26.078-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:25:26.078-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:25:26.078-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:25:26.274-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:25:26.390-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err receieved unexpected status code, 500, from update docs request for vb 10
.
CapiNozzle 2017-04-25T13:25:26.401-05:00 [INFO] Retrying update docs for vb 10 for the 2th time
CapiNozzle 2017-04-25T13:25:26.804-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:25:26.804-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:25:26.804-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:25:26.804-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:26.804-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:26.880-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:26.881-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:26.891-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:25:27.263-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err receieved unexpected status code, 500, from update docs request for vb 10
.
CapiNozzle 2017-04-25T13:25:27.274-05:00 [INFO] Retrying update docs for vb 10 for the 3th time
CapiNozzle 2017-04-25T13:25:29.257-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
PipelineManager 2017-04-25T13:25:29.258-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:25:29.258-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:25:29.258-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:29.258-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
PipelineRuntimeCtx 2017-04-25T13:25:29.258-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:25:29.259-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:29.259-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:25:29.259-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:29.259-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:25:29.259-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:25:29.259-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:25:29.259-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:25:29.259-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:29.259-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:25:29.259-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:29.259285818 -0500 CDT
GenericSupervisor 2017-04-25T13:25:29.259-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:29.259-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:25:29.259-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-34467063 has been stopped
GenericPipeline 2017-04-25T13:25:29.259-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-34467063 is stopped
PipelineManager 2017-04-25T13:25:29.259-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:25:29.259-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:17.709351454-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:25:29.259-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-34467063
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:29.260-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:25:29.260-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:29.260-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:25:29.260-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:29.260-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:25:29.260-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:29.260-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:25:29.260-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:29.260-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:29.260-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:25:29.261-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:29.260663651 -0500 CDT
DcpNozzle 2017-04-25T13:25:29.261-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:25:29.261-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:29.260858044 -0500 CDT
CapiNozzle 2017-04-25T13:25:29.262-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:25:29.262-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:29.262-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:29.262-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:29.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:25:29.262-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:29.262-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:25:29.264-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:29.264-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:29.264-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:29.264177474 -0500 CDT
DcpNozzle 2017-04-25T13:25:29.264-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:29.26418783 -0500 CDT
DcpNozzle 2017-04-25T13:25:29.264-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:25:29.264-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:25:29.264-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:29.264-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:25:29.264-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:29.264-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:25:29 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:25:29 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:25:29 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:25:29 Stream Closed for vb 1 StreamEnd simulated
CapiNozzle 2017-04-25T13:25:29.278-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:25:29.302-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:25:29.302-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:25:29.302-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:29.302-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:25:29.381-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:29.381-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:25:29.381-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:25:29.427-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:29.428-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:25:29.428-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:29.428-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:25:29.428-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:29.428-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:25:29.428-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:25:29.430-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:25:29.433-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
GenericSupervisor 2017-04-25T13:25:29.434-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:25:29.434-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
GenericSupervisor 2017-04-25T13:25:29.434-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:29.434-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:29.434-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:29.434-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:29.434-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:29.434-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
CheckpointManager 2017-04-25T13:25:29.434-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
CapiUtils 2017-04-25T13:25:29.464-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:29.464-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:29.464-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:25:29.464-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
StatisticsManager 2017-04-25T13:25:29.465-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:25:29.465-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
XDCRFactory 2017-04-25T13:25:29.465-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:25:29.465-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904 is constructed, start it
GenericPipeline 2017-04-25T13:25:29.466-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[131 148 7 33 107 138 145 86 112 113 139 158 117 149 61 106 160 134 169 60 67 96 129 153 163 4 30 31 93 11 64 80 109 17 42 97 150 9 18 70 71 72 102 45 91 48 53 78 2 26 116 161 94 98 52 21 59 135 164 27 62 66 6 49 68 121 132 152 159 69 95 128 162 144 146 39 105 89 110 16 37 141 32 55 73 79 23 43 65 133 15 29 47 50 108 119 57 165 8 56 83 151 58 84 123 126 143 111 22 38 92 137 167 168 12 54 36 82 87 88 81 90 46 140 157 170 5 74 114 118 124 127 14 10 13 130 40 120 122 125 0 34 35 75 101 19 24 154 28 51 99 1 44 104 136 25 63 155 156 100 115 142 166 20 41 85 76 103 3 77 147] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[216 304 196 330 334 238 180 245 278 336 305 319 183 194 232 328 185 317 249 256 288 215 228 199 211 300 227 235 212 244 265 248 299 182 209 338 298 198 221 233 333 253 193 257 172 175 190 284 287 339 178 224 292 308 322 303 202 220 230 240 273 297 302 295 255 260 271 173 229 272 325 207 294 326 277 200 247 291 301 321 332 252 179 186 237 243 231 309 315 195 275 306 340 197 219 320 323 289 318 208 214 258 269 331 241 266 201 234 290 316 296 189 205 274 312 313 225 263 279 293 236 254 176 217 270 310 327 203 239 280 307 174 187 206 213 329 286 314 191 246 171 283 324 177 226 268 281 251 204 210 261 192 264 282 311 222 262 285 188 218 250 335 181 276 337 184 267 223 242 259]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[104 136 25 63 155 156 100 115 142 166 20 41 85 76 103 3 77 147 131 148 7 33 107 138 145 86 112 113 139 158 117 149 61 106 160 134 163 169 60 67 96 129 153 4 30 31 93 11 64 80 109 17 42 97 150 9 18 70 71 72 102 45 91 48 53 78 2 26 116 161 94 98 52 21 59 135 164 27 62 66 6 159 49 68 121 132 152 69 95 128 162 144 146 39 105 89 110 16 37 141 32 55 73 79 23 43 65 133 15 29 47 50 108 119 57 165 8 56 83 151 58 84 123 126 143 111 168 22 38 92 137 167 12 54 36 82 87 88 81 90 46 140 127 157 170 5 74 114 118 124 14 10 13 130 40 120 122 125 0 34 35 75 101 19 24 154 28 51 99 1 44] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[222 262 285 218 188 335 250 181 276 337 184 267 259 223 242 196 216 304 330 334 238 180 245 278 336 232 305 319 183 194 328 185 317 249 256 288 211 215 228 199 300 227 235 265 212 244 209 248 299 182 338 198 298 221 233 333 253 257 193 172 175 339 190 284 287 322 178 224 292 308 273 303 202 220 230 240 297 302 271 295 255 260 173 229 272 325 207 294 326 247 277 200 332 291 301 321 243 252 179 186 237 231 309 315 275 195 306 340 197 219 320 323 269 289 318 208 214 258 331 234 241 266 201 290 316 189 296 205 274 312 293 313 225 263 279 217 236 254 176 270 310 327 307 203 239 280 174 187 206 213 329 286 314 191 246 171 283 268 324 177 226 251 281 261 204 210 311 192 264 282]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi active:true checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 stats_interval:1000 filter_expression: worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:25:29.466-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904...
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:29.476-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
StatisticsManager 2017-04-25T13:25:29.476-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:25:29.501-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:29.501-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:29.501-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:29.504-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:25:29.504-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:25:29.504-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:25:29.504-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CheckpointManager 2017-04-25T13:25:29.504-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:29.504-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:25:29.504-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:25:29.504-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:25:29.504-05:00 [INFO] Checkpointing starts in 1800.03 sec
PipelineRuntimeCtx 2017-04-25T13:25:29.504-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:25:29.504-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:25:29.505-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:25:29.505-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:25:29.507-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
StatisticsManager 2017-04-25T13:25:29.585-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:25:29.588-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:25:29.611-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:25:29.648-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:25:29.648-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:25:29.648-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:25:29.666-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:29.666-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:29.750-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:25:29.750-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:29.750-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:25:29.750-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:25:29.750-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:25:29.750-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:25:29.858-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:25:29.858-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:25:29.858-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:25:29.885-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:29.885-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:30.051-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:25:30.051-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:30.051-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:25:30.071-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:25:30.071-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:25:30.071-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:25:30.071-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904 is started----------
PipelineManager 2017-04-25T13:25:30.071-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:25:30.072-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:25:30.072-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:25:30.567-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:25:31.573-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:25:31.873-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:25:31.873-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:25:31.873-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:25:31.873-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:25:31 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:25:31 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:25:31.988-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:25:31.997-05:00 [INFO] Send batch count=51
2017/04/25 13:25:32 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:25:32.057-05:00 [INFO] Send batch count=74
2017/04/25 13:25:32 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 315 successful
CapiNozzle 2017-04-25T13:25:32.250-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:25:32 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 23 successful
CapiNozzle 2017-04-25T13:25:32.284-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:25:32 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 327 successful
CapiNozzle 2017-04-25T13:25:32.436-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:32.462-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:25:32 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 333 successful
StatisticsManager 2017-04-25T13:25:32.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:25:32 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 139 successful
CapiNozzle 2017-04-25T13:25:32.620-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:32.640-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:25:32 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 152 successful
CapiNozzle 2017-04-25T13:25:32.769-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:32.769-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:25:32.769-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:25:32.769-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:25:32.769-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
2017/04/25 13:25:32 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 255 successful
PipelineManager 2017-04-25T13:25:32.789-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:25:32.789-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:25:32.794-05:00 [INFO] err_list=[{"time":"2017-04-25T13:25:32.794202116-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:25:32.794-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:25:32 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 70 successful
CapiNozzle 2017-04-25T13:25:32.863-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:32.863-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:25:32 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 266 successful
CapiNozzle 2017-04-25T13:25:32.891-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:25:32 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:25:32 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 191 successful
CheckpointManager 2017-04-25T13:25:33.108-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
CheckpointManager 2017-04-25T13:25:33.108-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:25:33.160-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:25:33 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:25:33 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:25:33.204-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:33.229-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
DcpNozzle 2017-04-25T13:25:33.288-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:25:33.504-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:25:33.504-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904 {"Errors": "[{\"time\":\"2017-04-25T13:25:32.794202116-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 53, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 17486, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 53, "set_filtered": 0, "set_received_from_dcp": 17486, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:25:33.544-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:33.565-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:25:33.798-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:25:33.798-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:25:33.798-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:25:33.798-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:33.798-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:33.859-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:33.859-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:33.882-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:25:34.207-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:25:34.241-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:25:34.513-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:25:34.528-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:25:34.868-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:25:34.868-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:25:34.868-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:25:34.868-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:34.868-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:34.924-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:34.924-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:34.944-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:25:35.234-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:25:35.262-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:25:35.703-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:25:35.727-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:25:36.104-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:25:36.104-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:25:36.104-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:25:36.104-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:36.104-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:36.174-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:36.174-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:36.191-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:25:36.550-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:36.572-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:25:37.022-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:37.063-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:25:37.449-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:37.449-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:25:37.449-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:25:37.449-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:37.449-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:37.625-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:37.625-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:37.653-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:25:38.058-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:38.078-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:25:38.492-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:38.507-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:25:38.838-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:38.838-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:25:38.838-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:25:38.838-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:38.838-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:38.907-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:38.908-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:38.933-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:25:39.340-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:39.352-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
GenericSupervisor 2017-04-25T13:25:39.505-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:25:39.505-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:25:39.505-05:00 [INFO] Finish executing 0x681fb0
PipelineManager 2017-04-25T13:25:39.652-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:32.794202116-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
ToplogyChangeDetector 2017-04-25T13:25:39.687-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:25:40.173-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:40.184-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:25:40.919-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:25:40.919-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:25:40.919-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:25:40.919-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:40.920-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:40.989-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:40.989-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:40.999-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:25:41.628-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:25:41.639-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:25:42.012-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:25:42.025-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:25:42.175-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:25:42.186-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:25:42.395-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:42.406-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:25:42.794-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:25:42.794-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:25:42.794-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:42.794-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
PipelineRuntimeCtx 2017-04-25T13:25:42.794-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:25:42.794-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:25:42.794-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:42.794-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:25:42.794-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:42.794755412 -0500 CDT
GenericSupervisor 2017-04-25T13:25:42.794-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:42.794-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:25:42.794-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:42.794-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:25:42.794-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:42.794-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:25:42.794-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:25:42.795-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:25:42.795-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-428492904 has been stopped
GenericPipeline 2017-04-25T13:25:42.795-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428492904 is stopped
PipelineManager 2017-04-25T13:25:42.795-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:25:42.795-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:32.794202116-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:25:42.795-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:42.795-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
GenericPipeline 2017-04-25T13:25:42.795-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
GenericPipeline 2017-04-25T13:25:42.795-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
GenericPipeline 2017-04-25T13:25:42.795-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-428492904
AsyncListener 2017-04-25T13:25:42.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:42.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:42.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:42.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:25:42.796-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:42.796-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:25:42.796-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:25:42.796-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:42.796-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:25:42.796-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:25:42.796-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:42.796-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:25:42.796-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:25:42.796-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:42.796-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:25:42.796-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:25:42.796-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:42.797-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:42.797-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:42.79652148 -0500 CDT
DcpNozzle 2017-04-25T13:25:42.797-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:25:42.797-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:42.796816405 -0500 CDT
DcpNozzle 2017-04-25T13:25:42.797-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:25:42.798-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:25:42.798-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:42.798-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:25:42.798-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:42.798-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:25:42.798-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:42.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:25:42.798-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:42.799-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:25:42 Stream Closed for vb 0 StreamEnd simulated
GenericPipeline 2017-04-25T13:25:42.799-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:42.799-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:25:42.803-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:42.804-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:42.804045744 -0500 CDT
DcpNozzle 2017-04-25T13:25:42.804-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:42.804-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:25:42.804-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:42.804255948 -0500 CDT
GenericPipeline 2017-04-25T13:25:42.804-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:42.804-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:25:42.804-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:25:42.804-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:42.804-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:25:42 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:25:42 Stream Closed for vb 172 StreamEnd simulated
XDCRFactory 2017-04-25T13:25:42.914-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:42.915-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:25:42.915-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:25:43.019-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:43.020-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:25:43.020-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:43.020-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:25:43.020-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:43.020-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:25:43.020-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:25:43.021-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:25:43.021-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
GenericSupervisor 2017-04-25T13:25:43.023-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:25:43.023-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
GenericSupervisor 2017-04-25T13:25:43.023-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:43.023-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:43.023-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:43.023-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:43.023-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:43.023-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
CheckpointManager 2017-04-25T13:25:43.023-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
CapiUtils 2017-04-25T13:25:43.128-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:43.128-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:43.128-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:25:43.129-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
StatisticsManager 2017-04-25T13:25:43.129-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:25:43.129-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
XDCRFactory 2017-04-25T13:25:43.129-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:25:43.129-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061 is constructed, start it
GenericPipeline 2017-04-25T13:25:43.131-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[158 73 84 88 118 133 19 9 159 35 64 66 99 110 120 80 142 157 162 114 117 121 45 54 78 109 143 33 72 37 39 42 140 163 10 90 27 49 146 165 125 21 25 52 96 123 17 24 145 31 53 144 106 108 136 28 38 70 122 119 62 100 23 77 102 116 132 167 103 137 141 149 13 44 65 166 32 75 82 154 71 83 124 18 129 152 128 131 91 94 15 55 104 69 153 113 41 58 76 86 98 2 34 48 97 57 63 115 164 170 7 11 20 151 50 85 8 29 92 134 169 43 89 22 74 93 30 130 156 161 5 59 148 51 81 95 4 60 12 67 155 56 79 127 135 139 68 112 101 126 168 61 87 107 16 138 150 14 26 46 160 40 111 147 0 1 3 36 47 6 105] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[176 263 181 184 208 283 334 335 188 253 268 221 259 340 272 246 304 200 279 251 229 252 314 199 245 190 285 320 301 317 264 309 339 193 235 322 207 197 211 300 324 330 267 313 233 265 230 198 249 269 278 315 217 286 292 183 302 332 336 195 321 175 294 333 219 297 298 293 307 308 214 256 310 331 209 194 213 277 275 311 319 178 261 329 173 227 299 204 226 266 280 327 202 288 205 238 270 234 262 215 177 222 255 291 318 186 295 306 182 325 273 316 201 247 257 258 323 189 206 287 218 303 328 180 223 232 237 172 191 240 241 210 228 242 337 174 187 254 171 290 203 296 224 243 271 225 239 248 244 282 289 179 185 192 281 284 260 274 305 236 338 231 312 326 216 250 196 212 220 276]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[207 211 300 197 267 313 324 330 233 265 198 230 249 269 278 315 217 286 292 183 302 332 336 195 321 294 333 175 219 297 298 214 256 293 307 308 209 310 331 194 213 277 178 261 275 311 319 329 173 227 299 204 226 266 280 202 288 327 205 238 270 215 234 262 177 222 255 318 291 182 186 295 306 325 273 316 201 247 257 258 189 206 287 323 218 180 303 328 172 191 223 232 237 240 241 210 174 187 228 242 337 254 290 171 203 296 224 243 225 239 271 248 244 282 185 192 281 284 289 179 236 260 274 305 338 312 326 231 196 212 216 250 220 276 176 263 181 184 208 188 253 268 283 334 335 221 259 272 340 304 246 200 279 251 229 252 314 199 190 245 285 320 264 301 317 309 193 235 322 339] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[145 53 144 31 108 136 106 38 70 122 28 119 100 62 77 102 116 132 23 167 103 141 149 137 44 65 166 13 32 82 154 75 83 124 71 129 152 18 131 128 94 91 55 104 15 153 69 58 76 86 98 113 41 34 48 97 2 63 115 164 170 57 11 20 151 7 85 50 29 92 134 169 8 89 43 74 93 22 130 156 161 30 59 148 5 81 95 51 60 4 67 155 12 79 127 135 139 56 112 68 126 168 101 87 107 61 138 150 16 26 46 160 14 111 147 40 1 3 36 47 0 105 6 84 88 118 133 158 73 19 159 9 64 35 99 110 120 66 142 157 162 80 117 121 114 54 78 109 143 45 72 33 39 42 140 163 37 90 10 49 146 165 27 125 25 52 96 21 123 24 17]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: active:true failure_restart_interval:10 log_level:Info stats_interval:1000 replication_type:capi checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:25:43.131-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061...
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:43.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:25:43.163-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:43.163-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:25:43.163-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:25:43.163-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:25:43.163-05:00 [INFO] Checkpointing starts in 1801.214 sec
PipelineRuntimeCtx 2017-04-25T13:25:43.163-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:25:43.163-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:25:43.164-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:25:43.164-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:25:43.166-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:25:43.166-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:25:43.166-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:25:43.166-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:25:43.169-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:25:43.184-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:43.184-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:43.184-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:25:43.220-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:25:43.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:25:43.238-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:25:43.240-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:25:43.240-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:25:43.240-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:25:43.240-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:43.240-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:43.335-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:25:43.335-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:43.335-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:25:43.335-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:25:43.335-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:25:43.335-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:25:43.373-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:25:43.373-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:25:43.373-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:25:43.385-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:43.385-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:25:43.428-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:25:43.540-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:25:43.540-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:43.540-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:25:43.578-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:25:43.578-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:25:43.578-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:25:43.578-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061 is started----------
PipelineManager 2017-04-25T13:25:43.578-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:25:43.578-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:25:43.578-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:25:43.609-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:25:43.609-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:25:43.609-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:25:43.609-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:25:43 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:25:43.693-05:00 [INFO] Send batch count=47
2017/04/25 13:25:43 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:25:43.740-05:00 [INFO] Send batch count=181
CapiNozzle 2017-04-25T13:25:43.749-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:43.749-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:25:43 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:25:43.780-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:43.780-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:25:43.801-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:25:43 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:25:43.818-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:25:43 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 111 successful
CapiNozzle 2017-04-25T13:25:43.871-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:43.883-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:25:43 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:25:43 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 35 successful
CapiNozzle 2017-04-25T13:25:44.155-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:25:44.174-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:25:44.194-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:25:44 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 229 successful
CapiNozzle 2017-04-25T13:25:44.215-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
StatisticsManager 2017-04-25T13:25:44.245-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:25:44 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 171 successful
CapiNozzle 2017-04-25T13:25:44.645-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:25:44.645-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:25:44.645-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:25:44.645-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:25:44.645-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
2017/04/25 13:25:44 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 81 successful
PipelineManager 2017-04-25T13:25:44.656-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:25:44.656-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:25:44.671-05:00 [INFO] err_list=[{"time":"2017-04-25T13:25:44.671962919-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:25:44.672-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:25:44 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 85 successful
CapiNozzle 2017-04-25T13:25:44.761-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:44.761-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:44.776-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:25:44 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 185 successful
CapiNozzle 2017-04-25T13:25:44.829-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:25:44.846-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:25:44 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:25:44.912-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
CheckpointManager 2017-04-25T13:25:44.912-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:25:44 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:25:44 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:25:44.978-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:25:45.046-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:25:45.173-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:25:45.173-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061 {"Errors": "[{\"time\":\"2017-04-25T13:25:44.671962919-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 20, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 97, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 21279, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 97, "set_filtered": 0, "set_received_from_dcp": 21279, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:25:45.554-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:45.571-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:25:45.705-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:25:45.706-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:25:45.706-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:25:45.706-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:45.706-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:45.832-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:45.833-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:45.847-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:25:46.461-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:46.478-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:25:46.696-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:46.718-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:25:47.349-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:25:47.349-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:25:47.349-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:25:47.349-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:47.349-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:47.450-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:47.450-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:47.463-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:25:47.596-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:47.613-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:25:48.332-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:48.344-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:25:48.532-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:25:48.532-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:25:48.532-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:25:48.532-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:48.532-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:25:48.653-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:48.653-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:25:48.680-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:25:49.094-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:49.109-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:25:49.314-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:25:50.126-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:25:50.127-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:25:50.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:25:50.127-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:25:50.127-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:25:53.537-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:25:53.569-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:50.883-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
GenericSupervisor 2017-04-25T13:25:54.805-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:25:54.806-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:25:54.806-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:25:54.806-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:25:54.806-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:25:54.806-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:25:54.806-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
PipelineRuntimeCtx 2017-04-25T13:25:54.806-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:25:54.806-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:25:54.806-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:25:54.808-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:25:54.820-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:54.820-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:54.820048081 -0500 CDT
GenericSupervisor 2017-04-25T13:25:54.841-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:25:54.842-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:25:54.842-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:54.843-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
GenericSupervisor 2017-04-25T13:25:53.889-05:00 [INFO] Pipeline is no longer running, exit.
CheckpointManager 2017-04-25T13:25:54.854-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:54.854-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:25:54.854-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:25:54.854-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:25:54.854-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-20732061 has been stopped
GenericPipeline 2017-04-25T13:25:54.855-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-20732061 is stopped
PipelineManager 2017-04-25T13:25:54.855-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:25:54.855-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:44.671962919-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:25:54.855-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:25:54.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:54.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:54.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:54.947-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:54.947-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:54.947-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:54.947-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:54.948-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:25:54.948-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:54.948-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:54.948-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:54.948-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.948-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.951-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.951-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:25:54.917-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:25:54.983-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:25:54.983-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
AsyncListener 2017-04-25T13:25:54.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:25:54.984-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:25:54.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:25:54.958-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:25:54.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:25:54.917-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
GenericPipeline 2017-04-25T13:25:54.917-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
GenericPipeline 2017-04-25T13:25:54.985-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:25:54.924-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-20732061
GenericPipeline 2017-04-25T13:25:54.985-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:54.985-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:25:54.985-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:54.985122176 -0500 CDT
GenericPipeline 2017-04-25T13:25:54.985-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:25:54.985-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:54.985-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:54.985461506 -0500 CDT
CapiNozzle 2017-04-25T13:25:54.986-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:25:54.986-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:25:54.986-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:25:54.986-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:54.986-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:25:54.987-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:54.987-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:54.987702285 -0500 CDT
DcpNozzle 2017-04-25T13:25:54.987-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:25:54.987-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:54.987-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:25:54.987-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:25:54.987-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:25:54.987-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:25:54.987-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:25:54.989-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:25:54.989-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:25:54.989038744 -0500 CDT
DcpNozzle 2017-04-25T13:25:54.989-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:25:54.989-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:54.989-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:25:54.995-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:25:55.073-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:25:55.073-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:25:55.085-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:25:55.086-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
2017/04/25 13:25:55 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:25:55 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:25:55 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:25:55 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:25:55 Stream Closed for vb 2 StreamEnd simulated
AsyncListener 2017-04-25T13:25:55.454-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:25:55.454-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:25:55.454-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
ReplicationManager 2017-04-25T13:25:57.355-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=1, changes_left=315507
XDCRFactory 2017-04-25T13:25:58.406-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:58.407-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:25:58.407-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
PipelineManager 2017-04-25T13:25:58.520-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:25:44.671962919-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={2 source nozzles have been constructed}
]
XDCRFactory 2017-04-25T13:25:58.537-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:25:58.538-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:25:58.538-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:58.538-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:25:58.538-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:25:58.538-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:25:58.538-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:25:58.538-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:25:58.538-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
GenericSupervisor 2017-04-25T13:25:58.540-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:25:58.540-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
GenericSupervisor 2017-04-25T13:25:58.540-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:58.540-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:58.540-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:58.540-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:25:58.540-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:58.540-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
CheckpointManager 2017-04-25T13:25:58.540-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
CapiUtils 2017-04-25T13:25:58.619-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:58.619-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:58.619-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:25:58.620-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
StatisticsManager 2017-04-25T13:25:58.620-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:25:58.620-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
XDCRFactory 2017-04-25T13:25:58.620-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:25:58.620-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195 is constructed, start it
GenericPipeline 2017-04-25T13:25:58.621-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[138 166 29 108 111 124 77 137 139 0 68 155 24 54 169 43 10 106 128 162 17 22 23 42 59 16 78 104 123 31 56 47 91 94 134 52 67 89 97 103 126 3 21 40 66 73 69 74 114 147 159 49 12 18 96 34 164 113 170 9 76 167 4 15 35 168 121 135 8 130 1 45 14 116 157 11 37 25 55 136 26 117 28 84 90 93 64 141 27 38 100 102 105 127 163 2 32 81 6 13 83 63 131 109 145 151 153 107 156 115 143 158 160 87 148 154 70 92 95 129 152 36 41 48 60 71 39 85 72 119 142 62 82 118 133 46 58 165 98 125 33 86 99 144 146 101 122 140 57 150 44 79 80 112 120 19 30 53 149 50 51 20 65 88 110 161 5 75 7 132 61] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[184 252 253 292 312 326 211 300 323 225 301 233 319 309 179 185 231 275 283 303 237 274 321 282 171 210 295 296 298 339 201 260 330 289 216 230 266 265 223 306 246 248 287 272 307 177 189 198 338 337 180 293 238 245 215 218 226 254 340 193 219 227 305 328 175 241 325 281 256 268 333 336 183 280 335 224 302 334 229 299 191 192 255 261 176 240 250 264 284 181 209 214 258 186 187 234 221 212 310 173 195 263 314 257 273 276 190 318 324 290 232 271 182 286 311 313 317 197 236 242 291 213 267 329 200 262 327 332 244 316 203 315 331 208 285 294 322 249 251 308 220 178 196 277 304 205 270 204 206 269 320 278 199 222 207 297 259 172 174 239 202 235 243 279 288 188 194 217 228 247]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[47 91 94 134 126 52 67 89 97 103 3 21 40 66 73 69 74 114 147 159 49 12 18 96 34 164 113 170 9 76 167 4 15 35 168 121 135 8 130 1 45 14 116 157 11 37 25 55 136 26 117 28 84 90 93 64 141 27 38 100 102 105 127 163 2 32 81 6 13 83 63 131 109 145 151 153 107 156 115 143 158 160 87 148 154 70 92 95 129 152 36 41 48 60 71 39 85 72 119 142 62 82 118 133 46 58 165 98 125 33 86 99 144 146 101 122 140 57 150 44 79 80 112 120 19 30 53 149 50 51 20 65 88 110 161 5 75 7 132 61 138 166 29 108 111 124 77 137 139 0 68 155 24 54 169 43 10 106 128 162 17 22 23 42 59 16 78 104 123 31 56] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[223 246 306 248 287 272 307 198 338 177 189 337 180 293 238 218 226 245 215 254 340 305 328 193 219 227 175 241 325 281 256 268 333 336 183 280 335 302 224 334 229 255 261 299 191 192 176 240 250 264 284 214 258 181 209 234 186 187 221 310 212 173 195 263 273 276 314 257 190 318 324 232 271 290 313 182 286 311 242 291 317 197 236 213 267 329 332 200 262 327 244 316 315 203 294 322 331 208 285 249 251 308 178 196 220 277 304 205 270 204 206 269 320 199 222 278 207 297 259 172 174 239 288 202 235 243 279 247 188 194 217 228 292 312 184 252 253 300 323 326 211 225 301 233 319 283 303 309 179 185 231 275 274 321 237 171 210 282 295 296 298 339 330 201 260 289 216 230 266 265]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048]
CheckpointManager 2017-04-25T13:25:58.621-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195...
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:58.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:58.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:58.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:58.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:25:58.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:25:58.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
CheckpointManager 2017-04-25T13:25:58.638-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:25:58.638-05:00 [INFO] Checkpointing starts in 1801.021 sec
PipelineRuntimeCtx 2017-04-25T13:25:58.638-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:25:58.638-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:25:58.638-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:25:58.689-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:25:58.689-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:25:58.689-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:25:58.758-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:25:58.758-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:25:58.758-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:25:58.758-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:25:58.758-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:25:58.758-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:25:58.792-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:25:58.793-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:25:58.794-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:25:58.800-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:25:58.824-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:25:58.824-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:25:58.825-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
GenericSupervisor 2017-04-25T13:25:58.846-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:25:58.846-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
DcpNozzle 2017-04-25T13:25:58.846-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:25:58.846-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:25:58.907-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:25:58.910-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:25:59.788-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:25:59.790-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:25:59.790-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:25:59.790-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:25:59.790-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:25:59.790-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:25:59.791-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:25:59.806-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:25:59.806-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:25:59.806-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:25:59.826-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:25:59.826-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:25:59.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:26:00.126-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:26:00.126-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:26:00.126-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:26:00.126-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:26:00.127-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:26:00.127-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:26:00.127-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195 is started----------
PipelineManager 2017-04-25T13:26:00.127-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:26:00.127-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:26:00.127-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:26:00.210-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:26:00.210-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:26:00.210-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:26:00.215-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:26:00 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:26:00 UPR_STREAMREQ for vb 116 successful
StatisticsManager 2017-04-25T13:26:01.103-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:26:01 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 243 successful
StatisticsManager 2017-04-25T13:26:01.883-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:26:01 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:26:01 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 144 successful
CapiNozzle 2017-04-25T13:26:02.050-05:00 [INFO] Send batch count=11
2017/04/25 13:26:02 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 160 successful
CapiNozzle 2017-04-25T13:26:02.357-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
2017/04/25 13:26:02 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 270 successful
CapiNozzle 2017-04-25T13:26:02.373-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:26:02 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 164 successful
CapiNozzle 2017-04-25T13:26:02.422-05:00 [INFO] Send batch count=1
2017/04/25 13:26:02 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 278 successful
CapiNozzle 2017-04-25T13:26:02.450-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:02.463-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:26:02.490-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
2017/04/25 13:26:02 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:26:02.501-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CheckpointManager 2017-04-25T13:26:02.502-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
CheckpointManager 2017-04-25T13:26:02.502-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:26:02.518-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:02.518-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:26:02.518-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:26:02.518-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
ReplicationManager 2017-04-25T13:26:02.520-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
PipelineManager 2017-04-25T13:26:02.531-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:26:02.531-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:26:02.535-05:00 [INFO] err_list=[{"time":"2017-04-25T13:26:02.535900875-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:26:02.536-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:26:02 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:26:02 UPR_STREAMREQ for vb 295 successful
DcpNozzle 2017-04-25T13:26:02.591-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:26:02.622-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:02.622-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
DcpNozzle 2017-04-25T13:26:02.627-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:26:02.635-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
StatisticsManager 2017-04-25T13:26:02.758-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:26:02.759-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195 {"Errors": "[{\"time\":\"2017-04-25T13:26:02.535900875-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:26:03.044-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:03.056-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:26:03.484-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:03.495-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:26:03.827-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:03.837-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:26:03.837-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:26:03.837-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:03.845-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:04.637-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:04.718-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:04.746-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:26:07.314-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:26:07.328-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:26:08.450-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:26:08.463-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
GenericSupervisor 2017-04-25T13:26:08.807-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:26:08.813-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:26:08.813-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:26:08.972-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:26:09.002-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:26:09.002-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:26:09.003-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:09.086-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:26:09.678-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:26:02.535900875-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]}
]
CapiNozzle 2017-04-25T13:26:09.710-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:09.711-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:09.722-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
ToplogyChangeDetector 2017-04-25T13:26:09.903-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:26:10.575-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:26:10.588-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:26:11.026-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:26:11.046-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:26:11.388-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:26:11.388-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:26:11.388-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:26:11.388-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:11.388-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:11.489-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:11.489-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:11.501-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:26:11.942-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:26:11.953-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:26:12.286-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:26:12.297-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:26:12.423-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:26:12.439-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
PipelineManager 2017-04-25T13:26:12.541-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
PipelineManager 2017-04-25T13:26:12.541-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:26:12.541-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:26:12.541-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
PipelineRuntimeCtx 2017-04-25T13:26:12.541-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:26:12.541-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:26:12.541-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:26:12.541-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:12.541466382 -0500 CDT
GenericSupervisor 2017-04-25T13:26:12.541-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:26:12.541-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:26:12.541-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:26:12.541-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:26:12.541-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:26:12.541-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:26:12.541-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:26:12.541-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:26:12.541-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:26:12.541-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-538490195 has been stopped
GenericPipeline 2017-04-25T13:26:12.542-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-538490195 is stopped
PipelineManager 2017-04-25T13:26:12.542-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:26:12.542-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:26:02.535900875-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:26:12.542-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:26:12.542-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:26:12.542-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:26:12.542-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:26:12.542-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-538490195
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:26:12.542-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.542-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:26:12.542-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:26:12.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:26:12.543-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:26:12.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:12.543-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:12.543292306 -0500 CDT
DcpNozzle 2017-04-25T13:26:12.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:12.543555812 -0500 CDT
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:26:12.543-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:26:12.543-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:26:12.543-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:26:12.544-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:12.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:26:12.544-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:26:12.544-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:26:12.545-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:26:12.545-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:26:12.545-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:12.545443619 -0500 CDT
DcpNozzle 2017-04-25T13:26:12.545-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:12.545472969 -0500 CDT
DcpNozzle 2017-04-25T13:26:12.545-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:26:12.545-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:26:12.545-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:26:12.545-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:26:12.545-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:26:12.545-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:26:12 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:26:12 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:26:12 Stream Closed for vb 172 StreamEnd simulated
XDCRFactory 2017-04-25T13:26:12.651-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:26:12.652-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:26:12.652-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:26:12.696-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:26:12.697-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:26:12.697-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:26:12.697-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:26:12.697-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:26:12.697-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:26:12.697-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:26:12.697-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:26:12.698-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
GenericSupervisor 2017-04-25T13:26:12.700-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:26:12.700-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
GenericSupervisor 2017-04-25T13:26:12.700-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:12.700-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:12.700-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:12.700-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:12.700-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:26:12.700-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
CheckpointManager 2017-04-25T13:26:12.700-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
CapiUtils 2017-04-25T13:26:12.725-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:26:12.726-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:26:12.726-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:26:12.726-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
StatisticsManager 2017-04-25T13:26:12.727-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:26:12.727-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
XDCRFactory 2017-04-25T13:26:12.727-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:26:12.727-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700 is constructed, start it
GenericPipeline 2017-04-25T13:26:12.728-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[205 292 336 208 261 279 323 176 193 253 209 234 266 174 229 235 172 227 251 316 194 197 240 300 309 178 223 274 304 238 247 270 276 183 255 315 215 324 220 221 259 303 305 332 206 219 281 187 203 177 258 264 339 257 267 273 210 241 236 282 295 310 207 237 265 312 284 311 291 184 201 246 302 318 173 319 182 244 317 331 214 330 306 250 252 301 307 196 298 226 185 211 254 328 222 280 204 293 217 199 195 200 243 269 181 191 285 299 329 249 277 334 180 325 186 232 225 275 224 263 322 189 231 260 286 333 190 213 228 296 175 198 242 321 212 308 313 338 179 287 294 320 218 262 272 278 202 271 314 233 283 297 327 239 335 230 256 248 171 188 216 268 288 290 326 337 340 192 245 289] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[1 100 111 128 151 158 165 28 163 166 131 12 41 73 137 30 34 39 65 129 170 33 154 167 13 26 61 149 78 127 17 104 134 140 159 75 130 6 106 60 84 98 155 40 57 121 161 7 103 43 146 153 160 8 51 66 138 10 16 45 89 112 77 133 150 62 88 92 125 0 2 47 123 24 126 44 114 115 18 70 105 50 48 37 143 148 38 59 68 101 9 95 118 145 120 36 93 79 55 58 80 141 56 135 5 54 87 119 42 169 102 136 110 19 46 117 23 72 86 164 168 3 11 15 22 162 4 63 139 52 67 82 113 124 156 81 49 64 108 32 90 29 85 107 132 157 91 99 31 69 94 109 83 97 147 74 96 21 25 71 27 35 53 122 144 14 76 116 142 20 152]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[43 146 153 160 8 51 66 138 10 16 45 89 112 77 133 150 62 88 92 125 0 2 47 123 24 126 44 114 115 18 70 105 50 48 37 143 148 38 59 68 101 9 95 118 145 120 36 93 79 55 58 80 141 56 135 5 54 87 119 42 169 102 136 110 19 46 117 23 72 86 164 168 3 11 15 22 162 4 63 139 52 67 82 113 124 156 81 49 64 108 32 90 29 85 107 132 157 91 99 31 69 94 109 83 97 147 74 96 21 25 71 27 35 53 122 144 14 76 116 142 20 152 1 100 111 128 151 158 165 28 163 166 131 12 41 73 137 30 34 39 65 129 170 33 154 167 13 26 61 149 78 127 17 104 134 140 159 75 130 6 106 60 84 98 155 40 57 121 161 7 103] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[310 236 282 295 207 237 265 312 284 311 291 302 184 201 246 318 173 319 182 244 317 331 214 330 306 250 252 301 307 196 298 226 185 211 254 328 222 280 204 293 217 199 269 195 200 243 181 191 285 299 329 249 277 334 180 325 186 232 225 275 224 263 322 189 231 260 286 333 190 213 228 296 175 198 242 321 212 308 313 338 179 287 294 320 272 218 262 278 202 271 314 327 233 283 297 239 335 230 256 248 216 171 188 268 288 290 326 337 340 192 245 289 205 292 336 208 261 279 323 176 193 253 209 234 266 174 229 235 227 172 251 316 309 194 197 240 300 304 178 223 274 238 247 270 276 315 183 255 215 324 332 220 221 259 303 305 206 219 281 187 203 177 258 264 339 257 267 273 210 241]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 failure_restart_interval:10 source_nozzle_per_node:2 stats_interval:1000 target_nozzle_per_node:2 log_level:Info replication_type:capi active:true worker_batch_size:500 doc_batch_size_kb:2048 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:26:12.728-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700...
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:12.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:26:12.735-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:26:12.735-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:26:12.735-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:26:12.735-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:26:12.735-05:00 [INFO] Checkpointing starts in 1801.446 sec
PipelineRuntimeCtx 2017-04-25T13:26:12.735-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:26:12.735-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:26:12.735-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:26:12.735-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:26:12.750-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:26:12.750-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:26:12.750-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:26:12.750-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:26:12.750-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:26:12.750-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:26:12.751-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:26:12.755-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:26:12.758-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:26:12.781-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:26:12.781-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:26:12.781-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:26:12.798-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:26:12.805-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:26:12.805-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:26:12.850-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:26:12.857-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:26:12.927-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:26:12.928-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:26:12.928-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:26:12.929-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:26:12.929-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:26:12.941-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:26:12.987-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:26:12.987-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:26:12.987-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:26:13.014-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:26:13.014-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
AsyncListener 2017-04-25T13:26:13.080-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:26:13.080-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:26:13.080-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:26:13.144-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:26:13.144-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:26:13.144-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:26:13.171-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:26:13.172-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:26:13.172-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:26:13.172-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700 is started----------
PipelineManager 2017-04-25T13:26:13.172-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:26:13.172-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:26:13.172-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:26:13.423-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:26:13.423-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:26:13.423-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:26:13.423-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:26:13 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:26:13.584-05:00 [INFO] Send batch count=11
2017/04/25 13:26:13 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:26:13.630-05:00 [INFO] Send batch count=2
2017/04/25 13:26:13 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:26:13.638-05:00 [ERROR] Received malformed response from tcp connection vb=301. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:26:13.638-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err Received malformed response from tcp connection vb=301. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:26:13.658-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:26:13 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 329 successful
CapiNozzle 2017-04-25T13:26:13.781-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:26:13.789-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:26:13.793-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:26:13.806-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:26:13.841-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:26:13 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 39 successful
CapiNozzle 2017-04-25T13:26:13.872-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:26:13 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 247 successful
CapiNozzle 2017-04-25T13:26:13.904-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:26:13.920-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:26:13 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 149 successful
CapiNozzle 2017-04-25T13:26:13.963-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:26:13.963-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:26:13.963-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:26:13.963-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:26:13.963-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:26:13.978-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:26:13.978-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:26:13.980-05:00 [INFO] err_list=[{"time":"2017-04-25T13:26:13.980342611-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:26:13.980-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:26:13 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:26:13 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 67 successful
CapiNozzle 2017-04-25T13:26:14.066-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:14.066-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:26:14 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 271 successful
CapiNozzle 2017-04-25T13:26:14.087-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:26:14 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 93 successful
CheckpointManager 2017-04-25T13:26:14.360-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
CheckpointManager 2017-04-25T13:26:14.360-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:26:14 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:26:14 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:26:14.448-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:14.466-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
DcpNozzle 2017-04-25T13:26:14.474-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:26:14.660-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
DcpNozzle 2017-04-25T13:26:14.547-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:26:14.674-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
StatisticsManager 2017-04-25T13:26:14.751-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:26:14.828-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700 {"Errors": "[{\"time\":\"2017-04-25T13:26:13.980342611-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 4, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 3690, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 4, "set_filtered": 0, "set_received_from_dcp": 3690, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:26:15.562-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:15.564-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:26:15.610-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:26:15.632-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:26:15.989-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:26:15.989-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:26:15.989-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:26:15.989-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:15.989-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:16.137-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:16.230-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:16.547-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:16.842-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:26:16.842-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:26:17.400-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:26:17.622-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:26:17.629-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:26:17.629-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:26:17.634-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:17.643-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:17.674-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:26:17.803-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:17.803-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:17.819-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:26:18.693-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:26:18.715-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:26:18.743-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:26:18.763-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:26:19.280-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:26:19.280-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:26:19.280-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:26:19.280-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:19.280-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:19.391-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:19.391-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:19.408-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:26:19.514-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:26:19.525-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:26:19.746-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:26:19.759-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:26:20.050-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:26:20.051-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:26:20.051-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:26:20.051-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:20.051-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:20.109-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:20.109-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:20.120-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:26:20.192-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:26:20.203-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:26:20.438-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:26:20.456-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:26:20.579-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:26:20.580-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:26:20.580-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:26:20.580-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:20.580-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:20.701-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:20.701-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:20.716-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:26:20.744-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:26:20.777-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:26:21.084-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:26:21.098-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:26:21.270-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:26:21.301-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:26:21.301-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:26:21.330-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:21.873-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:26:21.875-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:26:21.900-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:26:21.972-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:26:21.976-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:26:21.986-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:26:22.806-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:26:22.806-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:26:22.806-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:26:22.806-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:26:22.806-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:26:22.806-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:26:22.810-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:26:22.810-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:26:22.810-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:26:22.824-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
ToplogyChangeDetector 2017-04-25T13:26:23.202-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:26:24.526-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
PipelineManager 2017-04-25T13:26:24.266-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:26:24.527-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:26:24.527-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:26:24.527-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
PipelineRuntimeCtx 2017-04-25T13:26:24.527-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:26:24.527-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:26:23.340-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
GenericSupervisor 2017-04-25T13:26:24.528-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:26:24.528-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:24.528015474 -0500 CDT
CapiNozzle 2017-04-25T13:26:24.528-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericSupervisor 2017-04-25T13:26:24.528-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:26:24.528-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:26:24.607-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:26:24.653-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CapiNozzle 2017-04-25T13:26:24.653-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CheckpointManager 2017-04-25T13:26:24.679-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:26:24.679-05:00 [INFO] Exits checkpointing routine.
CapiNozzle 2017-04-25T13:26:24.693-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
StatisticsManager 2017-04-25T13:26:24.699-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:26:24.699-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:26:24.711-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:26:24.711-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-697652700 has been stopped
GenericPipeline 2017-04-25T13:26:26.250-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
AsyncListener 2017-04-25T13:26:26.286-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:26.499-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:26.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:26.592-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:26.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:26.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:26.190-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:26.271-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:26.875-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:26:26.286-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
GenericPipeline 2017-04-25T13:26:26.889-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:26.401-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:26.437-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:27.052-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:27.085-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:27.160-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:26:26.122-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
AsyncListener 2017-04-25T13:26:26.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:26:27.942-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:27.951-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:26:26.173-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-697652700
GenericPipeline 2017-04-25T13:26:28.060-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:26:26.898-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:26:28.103-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:26:27.273-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:26:28.103-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:26:28.103-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:26:28.103-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:26:28.103-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:26:28.103-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:26:27.022-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:28.128-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:26:28.128-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:26:28.128-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:28.103255364 -0500 CDT
DcpNozzle 2017-04-25T13:26:28.129-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:26:28.129-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.129-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:26:28.129-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
DcpNozzle 2017-04-25T13:26:28.132-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:26:27.023-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:27.831-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:26:28.163-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:26:28.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:26:28.163-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:26:27.921-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:26:28.173-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-697652700 is stopped
PipelineManager 2017-04-25T13:26:28.173-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:26:28.173-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:26:13.980342611-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:26:28.173-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:26:27.951-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.174-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:26:27.967-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.174-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:26:28.174-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
DcpNozzle 2017-04-25T13:26:28.174-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:26:27.985-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:26:28.174-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:26:28.174-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:26:28.175-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:26:28.000-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:26:28.176-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:26:28.176-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:26:28.177-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:26:28.177-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:28.177638085 -0500 CDT
DcpNozzle 2017-04-25T13:26:28.177-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:26:28.034-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:26:28.178-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:26:28.178-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiNozzle 2017-04-25T13:26:28.064-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:26:28.178-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:26:26.465-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:26:28.186-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.186-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:26:28.220-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:28.178933722 -0500 CDT
AsyncListener 2017-04-25T13:26:28.220-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:26:28.220-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:26:28.220-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:26:28.220-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:26:28.296-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:26:28.297-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:26:28.297019559 -0500 CDT
DcpNozzle 2017-04-25T13:26:28.297-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:26:28.297-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:26:28.297-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:26:28.316-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:26:28.348-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:26:28.371-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:26:28.372-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:26:28.372-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:26:28.374-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
PipelineManager 2017-04-25T13:26:28.685-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:26:13.980342611-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:26:29.332-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
2017/04/25 13:26:29 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:26:29.529-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:26:29.849-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
2017/04/25 13:26:30 Stream Closed for vb 172 StreamEnd simulated
XDCRFactory 2017-04-25T13:26:30.337-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:26:32.003-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:26:32.937-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:26:32.938-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:26:32.943-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:26:32.943-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:26:33.205-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:26:33.311-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:26:33.421-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
GenericSupervisor 2017-04-25T13:26:33.618-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:26:33.624-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
GenericSupervisor 2017-04-25T13:26:33.624-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:33.624-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:33.624-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:33.624-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:33.624-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:26:33.624-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
CheckpointManager 2017-04-25T13:26:33.624-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
CapiUtils 2017-04-25T13:26:33.932-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:26:33.946-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:26:33.988-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:26:34.113-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
StatisticsManager 2017-04-25T13:26:34.148-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:26:34.157-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
XDCRFactory 2017-04-25T13:26:34.312-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:26:34.350-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 is constructed, start it
GenericPipeline 2017-04-25T13:26:34.880-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[44 55 94 164 41 149 105 26 6 32 92 160 1 144 145 165 3 82 65 96 120 159 0 169 106 151 117 5 114 127 33 89 100 112 47 40 115 25 42 81 142 61 163 35 98 110 132 9 17 60 10 52 104 15 154 71 136 79 21 30 122 20 74 134 72 59 67 139 36 88 28 77 86 87 8 54 85 118 53 119 76 147 148 58 43 38 138 23 75 101 146 12 124 157 83 46 45 116 64 166 140 162 49 51 141 24 66 84 95 111 125 62 70 109 137 22 48 143 4 131 126 170 113 103 129 167 97 161 158 39 29 18 37 56 11 135 156 91 73 99 168 19 57 16 78 153 31 7 155 2 69 107 63 27 34 68 13 123 108 128 133 93 50 80 102 121 130 152 14 150 90] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[286 196 265 186 227 237 283 278 287 308 319 293 217 339 322 328 334 268 282 199 242 315 178 216 300 188 253 249 284 316 255 173 239 313 194 228 248 190 264 266 329 195 275 176 241 243 327 210 258 335 180 206 218 323 280 302 312 314 231 236 261 277 317 207 200 212 215 189 247 254 304 310 201 203 272 281 285 309 325 311 187 191 204 222 238 330 245 226 177 294 260 298 326 331 181 185 279 305 197 202 292 318 174 182 208 193 250 340 184 256 259 270 296 235 257 263 214 233 246 291 171 179 324 274 172 183 205 211 273 338 288 306 209 219 230 295 337 303 251 269 336 271 301 220 321 234 307 223 192 232 262 320 175 225 252 267 289 213 198 224 276 297 221 290 299 332 333 240 229 244]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[327 176 241 243 210 258 335 206 218 323 180 302 312 314 280 277 317 207 231 236 261 215 200 212 189 247 254 304 310 281 285 309 201 203 272 311 325 191 204 222 238 187 245 330 226 177 294 260 298 326 331 181 185 279 305 292 318 197 202 174 182 208 193 340 250 259 270 296 184 256 263 235 257 214 291 233 246 171 179 274 324 172 183 205 211 273 338 288 306 209 219 230 295 303 337 336 251 269 271 301 321 220 234 307 223 232 262 192 320 225 252 267 289 175 213 224 276 297 198 221 333 240 290 299 332 229 244 286 186 196 265 227 237 283 319 278 287 308 293 217 339 322 328 334 268 282 199 242 315 178 216 300 188 253 284 316 249 255 239 313 173 194 228 248 329 190 264 266 195 275] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[163 132 9 35 98 110 10 17 60 15 52 104 71 154 79 136 20 21 30 122 72 74 134 59 36 67 139 88 87 8 28 77 86 53 54 85 118 119 58 76 147 148 38 43 138 146 12 23 75 101 83 124 157 45 46 64 116 140 166 49 162 24 51 141 111 125 62 66 84 95 22 70 109 137 4 48 143 131 113 126 170 97 103 129 167 158 161 29 39 11 18 37 56 91 135 156 19 73 99 168 16 57 31 78 153 2 7 155 63 69 107 13 27 34 68 108 123 93 128 133 121 130 152 14 50 80 102 90 150 44 41 55 94 164 149 26 105 160 1 6 32 92 3 144 145 165 65 82 0 96 120 159 106 169 151 5 117 33 114 127 47 89 100 112 25 40 115 42 81 61 142]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000 active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info]
CheckpointManager 2017-04-25T13:26:34.968-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079...
AsyncListener 2017-04-25T13:26:35.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:26:35.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
CapiUtils 2017-04-25T13:26:35.345-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:26:35.382-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
ToplogyChangeDetector 2017-04-25T13:26:35.418-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:26:35.481-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:26:35.512-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:26:35.517-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:26:35.517-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:26:35.517-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:26:35.537-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:26:35.540-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:26:35.600-05:00 [INFO] Checkpointing starts in 1802.451 sec
CheckpointManager 2017-04-25T13:26:36.070-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:26:36.023-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:26:36.934-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:26:37.109-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:26:37.356-05:00 [INFO] validateSourceTopology...
PipelineRuntimeCtx 2017-04-25T13:26:37.897-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:26:37.994-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:26:38.355-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
GenericSupervisor 2017-04-25T13:26:43.748-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:26:43.945-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:26:43.949-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:26:43.960-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:26:43.826-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:26:43.998-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:26:44.003-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CheckpointManager 2017-04-25T13:26:44.059-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:26:44.093-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:26:44.095-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:26:44.096-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:26:44.317-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:26:44.505-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:26:44.530-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:26:44.531-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
PipelineManager 2017-04-25T13:26:44.437-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:26:13.980342611-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:26:44.601-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:26:44.646-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
DcpNozzle 2017-04-25T13:26:44.591-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
CheckpointManager 2017-04-25T13:26:44.746-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:26:44.755-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:26:44.769-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CapiNozzle 2017-04-25T13:26:44.771-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:26:44.771-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:26:44.771-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:26:44.793-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:26:44.794-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:26:44.794-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:26:45.268-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:26:45.268-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:26:45.270-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:26:45.278-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:26:45.278-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:26:45.279-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:26:45.292-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 is started----------
PipelineManager 2017-04-25T13:26:45.319-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:26:45.319-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
StatisticsManager 2017-04-25T13:26:45.324-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:26:45.324-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:26:45 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:26:45 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:26:46 UPR_STREAMREQ for vb 330 successful
CapiNozzle 2017-04-25T13:26:45.905-05:00 [INFO] Send batch count=500
2017/04/25 13:26:46 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:26:46 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:26:46 UPR_STREAMREQ for vb 225 successful
GenericSupervisor 2017-04-25T13:26:46.411-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:26:46.556-05:00 [INFO] Send batch count=129
2017/04/25 13:26:48 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:26:49 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:26:50 UPR_STREAMREQ for vb 28 successful
GenericSupervisor 2017-04-25T13:26:49.641-05:00 [INFO] Executing 0x681fb0 timed out
2017/04/25 13:26:51 UPR_STREAMREQ for vb 29 successful
GenericSupervisor 2017-04-25T13:26:51.059-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:26:51.691-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:26:51 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:26:51 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 332 successful
StatisticsManager 2017-04-25T13:26:52.328-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:26:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 127 successful
StatisticsManager 2017-04-25T13:26:52.481-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:26:52 UPR_STREAMREQ for vb 31 successful
StatisticsManager 2017-04-25T13:26:52.528-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:26:13.980342611-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 681, "dcp_dispatch_time": 18, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 377, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 14567, "docs_rep_queue": 6723, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 10571, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 377, "set_filtered": 0, "set_received_from_dcp": 14567, "size_rep_queue": 9255406, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 5327}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 321, "docs_opt_repd": 0, "docs_rep_queue": 4683, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 321, "size_rep_queue": 6534914, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 415, "max": 5334, "mean": 5332.055421686747, "min": 5331}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 56, "docs_opt_repd": 0, "docs_rep_queue": 2040, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 56, "size_rep_queue": 2720492, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 105, "max": 5311, "mean": 5310.695238095238, "min": 5310}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 7570, "max": 15962, "mean": 25.508, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 9047, "expiry_received_from_dcp": 0, "set_received_from_dcp": 9047}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 681, "dcp_dispatch_time": {"count": 6997, "max": 123, "mean": 11.225, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 9603, "expiry_received_from_dcp": 0, "set_received_from_dcp": 9603}}

StatisticsManager 2017-04-25T13:26:52.529-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 9537 items, sent 500 items
StatisticsManager 2017-04-25T13:26:52.529-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 9603 items, sent 129 items
2017/04/25 13:26:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 230 successful
StatisticsManager 2017-04-25T13:26:52.567-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:9982]
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 9982 items, sent 9982 items. streams inactive: [86 89 151 79 74 98 141 170 72 81 132 63 64 88 96 162 167 61 93 97 37 59 50 165 168 33 41 87 92 95 65 68 83 129 149 66 78 147 148 157 163 128 138 155 47 160 55 82 145 42 91 153 32 58 84 164 57 136 142 51 70 75 85 34 39 48 130 156 36 43 45 150 161 38 44 77 94 54 60 80 134 152 35 53 71 49 69 90 135 52 40 67 62 158 159 76 146 166 137 144 154 131 143 99 133 140 46 56 169 73 139]
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:10080]
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 10080 items, sent 10080 items. streams inactive: [182 196 259 174 181 338 195 246 262 271 278 185 194 336 282 285 295 241 292 242 267 248 197 233 173 188 191 234 252 175 189 337 172 193 281 235 261 250 279 180 333 190 244 257 266 286 340 231 289 251 334 339 183 236 269 176 187 238 260 291 298 230 255 256 263 268 287 297 232 272 198 253 254 184 277 294 258 270 239 245 293 284 274 283 243 273 275 171 237 265 276 288 335 240 296 264 280 177 192 247 290 299 186 249 178 179 199]
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =445 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =445 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:26:52.569-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
2017/04/25 13:26:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:26:52 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 243 successful
StatisticsManager 2017-04-25T13:26:53.047-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:26:53 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 47 successful
CapiNozzle 2017-04-25T13:26:53.102-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
2017/04/25 13:26:53 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 247 successful
CapiNozzle 2017-04-25T13:26:53.248-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
2017/04/25 13:26:53 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:26:53 UPR_STREAMREQ for vb 248 successful
ToplogyChangeDetector 2017-04-25T13:26:56.387-05:00 [INFO] validateSourceTopology...
2017/04/25 13:26:56 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:26:56 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:26:56 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:26:56 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:26:56 UPR_STREAMREQ for vb 146 successful
GenericSupervisor 2017-04-25T13:26:56.537-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:27:03.670-05:00 [INFO] Executing 0x681fb0 timed out
CapiNozzle 2017-04-25T13:27:03.664-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: i/o timeout
2017/04/25 13:27:06 UPR_STREAMREQ for vb 249 successful
CapiNozzle 2017-04-25T13:27:05.936-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: i/o timeout.
GenericSupervisor 2017-04-25T13:27:06.060-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:27:05.150-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err read tcp 10.202.98.33:9091: i/o timeout.
GenericSupervisor 2017-04-25T13:27:07.741-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:27:08.178-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:27:08.937-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:27:08.973-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:27:09.009-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:27:09.033-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:27:09 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:27:09 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:27:09 UPR_STREAMREQ for vb 250 successful
DcpNozzle 2017-04-25T13:27:09.751-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
DcpNozzle 2017-04-25T13:27:10.021-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
GenericSupervisor 2017-04-25T13:27:09.981-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:27:10.108-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:27:11.196-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:27:11.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:27:11 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:27:11 UPR_STREAMREQ for vb 53 successful
StatisticsManager 2017-04-25T13:27:15.012-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:26:13.980342611-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1999, "dcp_dispatch_time": 10138, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 377, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 34631, "docs_rep_queue": 33962, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 7224, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 377, "set_filtered": 0, "set_received_from_dcp": 34631, "size_rep_queue": 41034297, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 5327}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 321, "docs_opt_repd": 0, "docs_rep_queue": 19815, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 321, "size_rep_queue": 24729795, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 415, "max": 5334, "mean": 5332.055421686747, "min": 5331}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 56, "docs_opt_repd": 0, "docs_rep_queue": 20832, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 56, "size_rep_queue": 26157686, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 105, "max": 5311, "mean": 5310.695238095238, "min": 5310}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 17170, "max": 6706936, "mean": 8254.341, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 20438, "expiry_received_from_dcp": 0, "set_received_from_dcp": 20438}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 73, "dcp_dispatch_time": {"count": 17459, "max": 7058358, "mean": 11990.909, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 20961, "expiry_received_from_dcp": 0, "set_received_from_dcp": 20961}}

2017/04/25 13:27:15 UPR_STREAMREQ for vb 148 successful
StatisticsManager 2017-04-25T13:27:15.237-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 22329 items, sent 129 items
StatisticsManager 2017-04-25T13:27:15.246-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 21956 items, sent 500 items
2017/04/25 13:27:16 UPR_STREAMREQ for vb 54 successful
GenericSupervisor 2017-04-25T13:27:16.167-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:27:16.872-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:22091]
StatisticsManager 2017-04-25T13:27:16.887-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 22092 items, sent 22091 items. streams inactive: [80 94 54 60 71 152 69 90 67 62 158 159 76 166 154 56 99 169 73 79 86 89 151 74 98 170 64 72 81 63 61 88 96 162 167 59 93 97 165 168 83 87 92 95 65 68 149 66 78 163 157 155 160 55 82 58 91 153 84 164 57 70 75 85 156 77 150 161]
StatisticsManager 2017-04-25T13:27:16.888-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:22799]
StatisticsManager 2017-04-25T13:27:16.889-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 22800 items, sent 22799 items. streams inactive: [183 269 176 187 260 291 298 255 256 263 268 287 297 272 198 253 254 184 277 294 258 270 293 284 274 283 273 275 171 265 276 288 296 264 280 177 192 290 299 186 178 179 199 182 196 259 174 181 195 262 271 278 185 194 282 285 295 292 267 197 173 188 191 252 175 189 172 193 281 261 279 180 190 257 266 286 289]
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =363 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =230 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:27:17.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
CapiNozzle 2017-04-25T13:27:17.068-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:27:17 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:27:18 UPR_STREAMREQ for vb 252 successful
CapiNozzle 2017-04-25T13:27:18.013-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
GenericSupervisor 2017-04-25T13:27:17.994-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:27:18.361-05:00 [INFO] ****************************
StatisticsManager 2017-04-25T13:27:21.153-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:27:21.241-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:27:21.242-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
2017/04/25 13:27:21 UPR_STREAMREQ for vb 55 successful
CapiNozzle 2017-04-25T13:27:22.465-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
ToplogyChangeDetector 2017-04-25T13:27:25.501-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:27:25.865-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:26:13.980342611-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 168, "dcp_dispatch_time": 1881, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 377, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 45472, "docs_rep_queue": 44261, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 10841, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 377, "set_filtered": 0, "set_received_from_dcp": 45472, "size_rep_queue": 56144699, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 5327}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 321, "docs_opt_repd": 0, "docs_rep_queue": 24639, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 321, "size_rep_queue": 31545469, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 415, "max": 5334, "mean": 5332.055421686747, "min": 5331}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 56, "docs_opt_repd": 0, "docs_rep_queue": 25282, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 56, "size_rep_queue": 32131180, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 105, "max": 5311, "mean": 5310.695238095238, "min": 5310}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 999, "dcp_dispatch_time": {"count": 22321, "max": 2045298, "mean": 3596.313, "min": 6}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 26007, "expiry_received_from_dcp": 0, "set_received_from_dcp": 26007}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 752, "dcp_dispatch_time": {"count": 23149, "max": 205189, "mean": 227.26, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 26047, "expiry_received_from_dcp": 0, "set_received_from_dcp": 26047}}

2017/04/25 13:27:25 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:27:25 UPR_STREAMREQ for vb 56 successful
GenericSupervisor 2017-04-25T13:27:25.733-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:27:25.883-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
StatisticsManager 2017-04-25T13:27:26.094-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 26813 items, sent 500 items
StatisticsManager 2017-04-25T13:27:26.094-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 27624 items, sent 129 items
StatisticsManager 2017-04-25T13:27:26.099-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:26813]
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 26813 items, sent 26813 items. streams inactive: [64 72 81 63 61 88 96 162 167 59 93 97 165 168 83 87 92 95 65 68 66 78 163 157 155 160 82 58 91 153 84 164 57 70 75 85 156 77 161 80 94 60 71 152 69 90 67 62 158 159 76 166 154 99 169 73 79 86 89 151 74 98 170]
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:27624]
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 27624 items, sent 27624 items. streams inactive: [274 283 171 273 275 265 276 288 264 280 296 177 192 186 290 299 178 179 199 174 181 182 196 259 278 185 194 195 262 271 282 285 295 267 292 197 173 188 175 189 191 172 193 261 281 180 279 190 257 266 286 289 183 176 187 269 260 291 298 287 255 256 263 268 297 198 272 184 253 254 258 270 277 294 293 284]
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =999 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =999 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =358 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =358 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:27:26.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
2017/04/25 13:27:26 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:27:28 UPR_STREAMREQ for vb 57 successful
GenericSupervisor 2017-04-25T13:27:27.508-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:27:26.253-05:00 [INFO] About to call function
2017/04/25 13:27:28 UPR_STREAMREQ for vb 151 successful
GenericSupervisor 2017-04-25T13:27:28.355-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:27:28.869-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
GenericSupervisor 2017-04-25T13:27:29.036-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:27:29 UPR_STREAMREQ for vb 152 successful
GenericSupervisor 2017-04-25T13:28:10.566-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:28:10.234-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
GenericSupervisor 2017-04-25T13:28:10.592-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:28:10.601-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:28:10.784-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:28:10.446-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
GenericSupervisor 2017-04-25T13:28:10.821-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:28:10.833-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:10.836-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:28:10.836-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:28:10.837-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]
ReplicationManager 2017-04-25T13:28:11.064-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]
2017/04/25 13:28:14 UPR_STREAMREQ for vb 254 successful
GenericSupervisor 2017-04-25T13:28:14.125-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:28:14.646-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:28:14.650-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:28:14.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:28:14.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
PipelineManager 2017-04-25T13:28:14.998-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:28:15.010-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:15.111-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:15.117-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:15.117-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:28:15.117-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:28:15.117-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:15.119-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:28:15.215-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:15.059344829-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]}
]
PipelineManager 2017-04-25T13:28:15.059-05:00 [INFO] err_list=[{"time":"2017-04-25T13:28:15.059344829-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]
PipelineManager 2017-04-25T13:28:15.223-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
ToplogyChangeDetector 2017-04-25T13:28:15.372-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:28:15 UPR_STREAMREQ for vb 58 successful
StatisticsManager 2017-04-25T13:28:15.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:28:15.536-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:28:15.536-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 {"Errors": "[{\"time\":\"2017-04-25T13:28:15.059344829-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1783, "dcp_dispatch_time": 717, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 377, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 57127, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 377, "set_filtered": 0, "set_received_from_dcp": 57127, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]", "Status": "Pending"}

2017/04/25 13:28:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:28:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:28:15 UPR_STREAMREQ for vb 59 successful
GenericSupervisor 2017-04-25T13:28:16.234-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:28:16.234-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:28:16.243-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:28:16 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:28:16 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:28:16 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:28:16 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:28:17 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:28:17 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 260 successful
DcpNozzle 2017-04-25T13:28:18.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 3
DcpNozzle 2017-04-25T13:28:18.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 3
2017/04/25 13:28:18 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:28:18 UPR_STREAMREQ for vb 262 successful
PipelineManager 2017-04-25T13:28:18.678-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:15.059344829-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries]}
]
CapiNozzle 2017-04-25T13:28:18.695-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:18.695-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:18.718-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:18.719-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:18.909-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:28:18.928-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:28:19 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:28:19 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:28:20 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:28:20 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:28:20 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:28:24 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:28:24 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:28:25 UPR_STREAMREQ for vb 265 successful
GenericSupervisor 2017-04-25T13:28:25.903-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:28:25.922-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:28:26.063-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:28:26.073-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:28:26 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 170 successful
PipelineManager 2017-04-25T13:28:26.124-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries 
PipelineManager 2017-04-25T13:28:26.126-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:28:26.129-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:28:26.131-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
PipelineRuntimeCtx 2017-04-25T13:28:26.133-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:28:26.135-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:26.136-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:28:26.141-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:26.135522353 -0500 CDT
GenericSupervisor 2017-04-25T13:28:26.148-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:26.148-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:28:26.149-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:28:26.150-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:28:26.150-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:28:26.150-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:28:26.151-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:28:26.151-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 has been stopped
2017/04/25 13:28:26 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 267 successful
GenericPipeline 2017-04-25T13:28:26.298-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-195316079 is stopped
PipelineManager 2017-04-25T13:28:26.298-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:28:26.299-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:15.059344829-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:28:26.303-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:28:26 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 171 successful
AsyncListener 2017-04-25T13:28:26.462-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:26.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:26.471-05:00 [INFO] About to call function
2017/04/25 13:28:26 UPR_STREAMREQ for vb 78 successful
AsyncListener 2017-04-25T13:28:26.532-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:26.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:26.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
2017/04/25 13:28:26 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:28:26 UPR_STREAMREQ for vb 269 successful
GenericPipeline 2017-04-25T13:28:26.460-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
ReplicationManager 2017-04-25T13:28:26.727-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
GenericPipeline 2017-04-25T13:28:26.461-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
GenericPipeline 2017-04-25T13:28:26.759-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:28:26.760-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:28:26.764-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:28:26.461-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
GenericPipeline 2017-04-25T13:28:26.783-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:28:26.784-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:28:26.785-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:28:26.785-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:26.764496877 -0500 CDT
DcpNozzle 2017-04-25T13:28:26.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
GenericPipeline 2017-04-25T13:28:26.461-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-195316079
GenericPipeline 2017-04-25T13:28:26.793-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:28:26.793-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:28:26.793-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:28:26.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:28:26.797-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:28:26.797-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
GenericPipeline 2017-04-25T13:28:26.798-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:28:26.798-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:28:26.798-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
CapiNozzle 2017-04-25T13:28:26.798-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:26.793793704 -0500 CDT
CapiNozzle 2017-04-25T13:28:26.799-05:00 [INFO] releasing capi client
DcpNozzle 2017-04-25T13:28:26.799-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:28:26.799-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:28:26.799-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:28:26.799-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
DcpNozzle 2017-04-25T13:28:26.799-05:00 [INFO] There is no active stream for vb=173
DcpNozzle 2017-04-25T13:28:26.799-05:00 [INFO] There is no active stream for vb=174
DcpNozzle 2017-04-25T13:28:26.799-05:00 [INFO] There is no active stream for vb=175
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=176
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=177
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=178
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=179
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=180
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=181
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=182
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=183
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=184
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=185
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=186
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=187
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=188
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=189
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=190
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=191
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=192
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=193
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=194
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=195
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=196
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=197
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=198
DcpNozzle 2017-04-25T13:28:26.800-05:00 [INFO] There is no active stream for vb=199
CapiNozzle 2017-04-25T13:28:26.800-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:28:26.800-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:26.800-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:28:26.800-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:26.800-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=270
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=271
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=272
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=273
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=274
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=275
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=276
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=277
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=278
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=279
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=280
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=281
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=282
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=283
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=284
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=285
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=286
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=287
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=288
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=289
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=290
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=291
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=292
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=293
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=294
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=295
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=296
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=297
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=298
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=299
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=79
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=80
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=81
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=82
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=83
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=84
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=85
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=86
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=87
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=88
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=89
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=90
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=91
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=92
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=93
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=94
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=95
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=96
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=97
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=98
DcpNozzle 2017-04-25T13:28:26.917-05:00 [INFO] There is no active stream for vb=99
DcpNozzle 2017-04-25T13:28:26.955-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:28:26.955-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:26.955607957 -0500 CDT
DcpNozzle 2017-04-25T13:28:26.955-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:28:26.955-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:26.955-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:28:26.969-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:28:26.969-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:26.969303718 -0500 CDT
DcpNozzle 2017-04-25T13:28:26.969-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:28:26.969-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:26.969-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
PipelineManager 2017-04-25T13:28:27.409-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:15.059344829-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 103 after 3 retries"}]}, progress={Start pipeline construction}
]
DcpNozzle 2017-04-25T13:28:27.818-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 4
XDCRFactory 2017-04-25T13:28:27.894-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:28:27.903-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:28:27.903-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
DcpNozzle 2017-04-25T13:28:28.134-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 5
DcpNozzle 2017-04-25T13:28:28.164-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 6
XDCRFactory 2017-04-25T13:28:28.213-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:28:28.216-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:28:28.218-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:28:28.218-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:28:28.219-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:28:28.219-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:28:28.219-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:28:28.226-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:28:28.231-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
GenericSupervisor 2017-04-25T13:28:28.237-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:28:28.237-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
GenericSupervisor 2017-04-25T13:28:28.238-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:28.238-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:28.238-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:28.238-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:28.238-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:28:28.243-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
CheckpointManager 2017-04-25T13:28:28.243-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
CapiUtils 2017-04-25T13:28:28.278-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:28:28.278-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:28:28.278-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:28:28.281-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
StatisticsManager 2017-04-25T13:28:28.291-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:28:28.292-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
XDCRFactory 2017-04-25T13:28:28.293-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:28:28.294-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374 is constructed, start it
GenericPipeline 2017-04-25T13:28:28.296-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[82 113 6 129 166 11 23 51 165 20 75 19 99 112 147 29 49 85 122 120 18 140 159 163 150 26 97 7 3 12 30 66 104 109 137 15 108 101 169 13 161 24 36 81 21 96 124 9 38 42 72 144 44 46 164 89 91 100 134 37 139 149 74 55 31 87 17 41 43 94 146 80 53 111 154 5 153 162 45 106 156 168 68 130 50 121 22 48 64 148 4 47 71 88 93 136 152 118 0 70 107 158 58 95 125 16 40 59 14 60 69 105 145 33 126 92 151 73 123 142 10 65 78 114 8 102 35 117 138 170 2 39 56 79 132 131 167 32 67 90 143 155 119 54 86 98 133 27 28 34 116 127 135 160 62 63 128 61 25 84 52 103 157 76 110 1 77 83 57 115 141] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[206 286 323 283 277 325 340 284 306 332 266 288 319 313 172 190 236 272 220 223 240 257 260 327 295 317 320 339 201 253 300 237 246 338 195 330 214 241 249 262 213 242 290 176 173 193 217 299 225 235 265 314 316 182 229 280 335 293 321 194 255 304 322 296 216 232 268 294 312 318 184 204 205 231 247 289 179 270 315 279 209 233 276 282 177 334 287 183 243 336 221 259 302 337 200 298 226 278 308 185 218 309 328 324 250 254 285 301 303 307 297 192 248 256 211 273 305 198 202 208 244 269 238 274 331 261 311 180 203 171 186 178 210 275 212 281 207 245 263 174 175 187 215 329 189 310 333 188 239 224 251 230 267 292 181 199 258 326 191 219 227 228 196 234 222 197 252 264 271 291]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[29 49 85 122 120 18 140 159 163 150 26 97 7 137 3 12 30 66 104 109 15 108 101 169 13 161 24 36 81 21 96 124 9 38 42 72 144 44 46 164 89 91 100 134 37 139 149 74 55 31 87 17 41 43 94 146 80 53 111 154 5 153 162 45 106 156 168 68 130 50 121 22 48 64 148 152 4 47 71 88 93 136 118 0 70 107 158 58 95 125 16 40 59 14 60 69 105 145 33 126 92 151 73 123 142 10 65 78 114 8 102 35 117 138 170 2 39 56 79 132 131 167 32 67 90 143 155 119 54 86 98 133 160 27 28 34 116 127 135 62 63 128 61 25 84 52 103 157 76 110 1 77 83 57 115 141 82 113 6 129 166 11 23 51 165 20 75 19 99 112 147] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[313 172 190 236 272 220 223 240 257 260 327 295 317 320 339 201 253 300 237 246 338 195 330 214 241 249 262 213 242 290 176 299 173 193 217 225 235 265 314 280 316 182 229 335 293 321 194 255 304 322 294 296 216 232 268 312 318 289 184 204 205 231 247 179 270 315 233 279 209 276 282 177 334 287 183 243 302 336 221 259 337 200 298 226 278 308 185 218 309 328 307 324 250 254 285 301 303 297 192 248 256 211 273 305 269 198 202 208 244 238 274 331 311 261 180 203 171 186 178 210 275 212 281 207 245 263 187 174 175 329 215 189 310 239 333 188 224 251 230 267 292 326 181 199 258 191 219 227 228 196 234 222 197 252 264 271 291 206 286 323 283 340 277 325 284 306 319 332 266 288]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 stats_interval:1000 filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 log_level:Info replication_type:capi checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:28:28.298-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374...
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:28:28.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:28.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:28.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:28.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:28:28.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
StatisticsManager 2017-04-25T13:28:28.315-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:28:28.323-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:28:28.323-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:28:28.323-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:28:28.323-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:28.323-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:28:28.324-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:28:28.324-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:28:28.329-05:00 [INFO] Checkpointing starts in 1803.695 sec
PipelineRuntimeCtx 2017-04-25T13:28:28.329-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:28:28.338-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:28:28.347-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:28:28.347-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:28:28.355-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:28:28.355-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CapiNozzle 2017-04-25T13:28:28.355-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:28:28.356-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:28:28.388-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:28:28.434-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:28:28.453-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:28:28.500-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:28:28.828-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:28:28.828-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:28:28.828-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:28:28.841-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:28:28.841-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:28:29.003-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:28:29.004-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:28:29.004-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:28:29.028-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:28:29.029-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:28:29.029-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:28:29.322-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:28:29.322-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:28:29.322-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:28:29.348-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:28:29.348-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:28:29.453-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:28:29.522-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:28:29.522-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:28:29.522-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:28:29.551-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:28:29.551-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:28:29.551-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:28:29.551-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374 is started----------
PipelineManager 2017-04-25T13:28:29.551-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:28:29.551-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:28:29.552-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:28:30.409-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:28:31.415-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:28:31.534-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:28:31.534-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:28:31.534-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:28:31.534-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:28:31 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 114 successful
CapiNozzle 2017-04-25T13:28:31.913-05:00 [INFO] Send batch count=66
2017/04/25 13:28:31 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:28:31 UPR_STREAMREQ for vb 312 successful
CapiNozzle 2017-04-25T13:28:31.982-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:32.024-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:28:32 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 26 successful
CapiNozzle 2017-04-25T13:28:32.147-05:00 [INFO] Send batch count=212
2017/04/25 13:28:32 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 322 successful
CapiNozzle 2017-04-25T13:28:32.191-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:32.192-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:32.197-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:32.207-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:28:32.209-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
2017/04/25 13:28:32 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 338 successful
CapiNozzle 2017-04-25T13:28:32.378-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:28:32.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:28:32.392-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:28:32 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 260 successful
CapiNozzle 2017-04-25T13:28:32.617-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:28:32.630-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
2017/04/25 13:28:32 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 282 successful
CapiNozzle 2017-04-25T13:28:32.702-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:32.702-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:28:32.702-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:28:32.702-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:28:32.725-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:28:32 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 81 successful
PipelineManager 2017-04-25T13:28:32.757-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:28:32.757-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:28:32.757-05:00 [INFO] err_list=[{"time":"2017-04-25T13:28:32.757203704-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:28:32.759-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:28:32 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 197 successful
CheckpointManager 2017-04-25T13:28:32.807-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
CheckpointManager 2017-04-25T13:28:32.807-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:28:32.829-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:32.829-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:28:32 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:28:32.840-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:28:32 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:28:32 UPR_STREAMREQ for vb 198 successful
CapiNozzle 2017-04-25T13:28:32.906-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:28:32.931-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
DcpNozzle 2017-04-25T13:28:32.942-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:28:32.951-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:28:33.113-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:33.129-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:28:33.266-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:28:33.266-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:28:33.266-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:28:33.266-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:33.266-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:28:33.330-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:28:33.330-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374 {"Errors": "[{\"time\":\"2017-04-25T13:28:32.757203704-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 115, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 4166, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 115, "set_filtered": 0, "set_received_from_dcp": 4166, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:28:33.364-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:33.365-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:33.375-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:28:33.750-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:33.789-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:28:34.068-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:28:34.087-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:28:34.177-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:34.177-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:28:34.177-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:28:34.177-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:34.177-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:34.259-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:34.259-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:34.277-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:28:34.380-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:28:34.392-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:28:34.670-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:28:34.685-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:28:34.811-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:28:34.811-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:28:34.811-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:28:34.811-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:34.811-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:34.872-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:34.872-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:34.883-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:28:35.112-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:28:35.127-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:28:35.385-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:28:35.397-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:28:35.580-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:28:35.580-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:28:35.580-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:28:35.580-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:35.580-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:35.673-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:35.673-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:35.684-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:28:35.923-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:28:35.934-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:28:36.102-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:28:36.114-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:28:36.366-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:28:36.366-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:28:36.366-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:28:36.366-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:36.366-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:36.519-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:36.519-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:36.531-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:28:36.790-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:28:36.841-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:28:37.288-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:28:37.303-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:28:37.412-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:28:37.412-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:28:37.412-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:28:37.412-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:37.412-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:37.513-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:37.513-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:37.528-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:28:37.712-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:28:37.729-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:28:37.911-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:37.923-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:28:38.075-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:28:38.075-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:28:38.075-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:28:38.075-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:38.075-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:38.166-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:38.166-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:38.185-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:28:38.308-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:38.326-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
GenericSupervisor 2017-04-25T13:28:38.380-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:28:38.398-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:28:38.407-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:28:38.544-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:28:38.603-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:28:38.640-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:28:38.891-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:38.891-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:28:38.891-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:28:38.891-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:38.891-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:38.975-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:38.975-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:39.013-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:28:39.270-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:28:39.295-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:28:39.482-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:28:39.494-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:28:39.809-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:28:39.809-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:28:39.809-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:28:39.809-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:39.809-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:28:39.831-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:32.757203704-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:28:39.899-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:39.899-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:39.921-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:28:40.090-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:28:40.105-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:28:40.477-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:40.530-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:28:40.693-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:28:40.718-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:28:40.719-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:28:40.720-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:40.744-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:41.092-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:41.399-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
PipelineManager 2017-04-25T13:28:42.816-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:28:42.930-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:28:42.934-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:28:42.775-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:42.958-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericPipeline 2017-04-25T13:28:42.940-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
PipelineRuntimeCtx 2017-04-25T13:28:42.997-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:28:43.000-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:43.041-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
CapiNozzle 2017-04-25T13:28:43.051-05:00 [INFO] Retrying update docs for vb 106 for the 1th time
GenericSupervisor 2017-04-25T13:28:43.047-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:43.001097567 -0500 CDT
GenericSupervisor 2017-04-25T13:28:43.140-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:43.140-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:28:43.142-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:28:43.144-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:28:43.145-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:28:43.145-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:28:43.150-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:28:43.150-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:28:43.151-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:28:43.151-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-219402374 has been stopped
GenericPipeline 2017-04-25T13:28:43.179-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-219402374 is stopped
PipelineManager 2017-04-25T13:28:43.179-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:28:43.180-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:32.757203704-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:28:43.184-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:28:43.203-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
GenericPipeline 2017-04-25T13:28:43.207-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
GenericPipeline 2017-04-25T13:28:43.207-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
GenericPipeline 2017-04-25T13:28:43.207-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-219402374
AsyncListener 2017-04-25T13:28:43.211-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:43.217-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:43.217-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:43.217-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:28:43.218-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:28:43.219-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:28:43.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:28:43.241-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:28:43.241-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.247-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.248-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.248-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.248-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.248-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:28:43.248-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:28:43.241-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:28:43.250-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:28:43.250-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:28:43.250-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:28:43.250-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:28:43.248-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.251-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:43.251-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.251-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:43.251-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.252-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:43.252-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.252-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:43.252-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.252-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:28:43.261-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:28:43.261-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:28:43.261-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:28:43.261-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:43.25068486 -0500 CDT
DcpNozzle 2017-04-25T13:28:43.227-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:28:43.272-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:28:43.272-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:43.261514618 -0500 CDT
CapiNozzle 2017-04-25T13:28:43.282-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:28:43.282-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:28:43.296-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:28:43.296-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:28:43.297-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:28:43.297-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:28:43.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:28:43.297-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:43.298-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:28:43.299-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:43.299-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:28:43.322-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:28:43.323-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:43.323029663 -0500 CDT
DcpNozzle 2017-04-25T13:28:43.323-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:28:43.323-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:43.323-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:28:43.324-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:28:43.325-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:43.32497543 -0500 CDT
DcpNozzle 2017-04-25T13:28:43.325-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:28:43.325-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:28:43.325-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
XDCRFactory 2017-04-25T13:28:43.454-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:28:43.459-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:28:43.459-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:28:43.637-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:28:43.641-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:28:43.646-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:28:43.646-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:28:43.646-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:28:43.646-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:28:43.646-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:28:43.649-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:28:43.654-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
GenericSupervisor 2017-04-25T13:28:43.663-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:28:43.663-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
GenericSupervisor 2017-04-25T13:28:43.664-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:43.664-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:43.664-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:43.664-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:28:43.664-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:28:43.669-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
CheckpointManager 2017-04-25T13:28:43.669-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
CapiUtils 2017-04-25T13:28:43.789-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:28:43.789-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:28:43.790-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:28:43.799-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
StatisticsManager 2017-04-25T13:28:43.811-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:28:43.812-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
XDCRFactory 2017-04-25T13:28:43.813-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:28:43.814-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202 is constructed, start it
GenericPipeline 2017-04-25T13:28:43.816-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[190 273 286 255 276 319 333 246 180 279 287 200 221 269 297 309 209 237 284 295 296 314 338 188 187 230 330 264 223 267 192 272 289 248 182 214 236 240 323 253 174 194 234 313 320 324 225 312 224 280 288 306 177 249 282 335 202 254 268 185 220 334 300 308 327 329 206 266 196 207 217 229 210 259 283 228 258 322 176 179 197 270 204 189 215 278 290 175 239 292 328 173 201 257 277 326 332 291 171 186 235 261 198 265 285 275 339 183 191 181 212 262 304 195 243 247 252 274 294 303 250 232 317 219 222 233 241 331 227 184 199 337 178 260 299 318 226 238 281 301 316 336 205 211 340 218 256 208 213 302 245 251 203 231 244 325 216 293 311 305 321 172 193 242 271 315 298 307 310 263] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[74 46 100 111 115 167 37 102 112 131 81 119 30 95 107 21 99 132 146 73 85 168 50 61 72 44 39 142 160 15 8 67 109 5 32 77 31 152 41 47 54 68 86 114 155 43 69 13 148 2 169 59 64 159 6 124 135 143 93 56 65 141 28 90 126 45 75 76 103 58 134 156 117 110 22 105 57 52 129 144 0 83 11 138 149 16 151 137 127 104 140 147 157 18 55 62 108 53 89 136 153 71 154 158 162 97 78 118 60 42 113 25 7 106 122 3 23 9 116 12 17 66 80 84 161 170 1 14 48 166 29 34 82 87 139 163 4 26 88 120 20 96 40 38 125 145 24 36 79 121 165 10 130 164 33 92 128 35 91 94 70 49 19 123 133 150 101 27 63 98 51]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[263 298 307 310 190 273 286 246 255 276 319 333 180 279 287 200 221 269 297 309 209 237 284 338 295 296 314 188 330 187 230 264 223 267 192 272 289 248 182 214 236 240 323 253 320 174 194 234 313 324 225 312 288 306 224 280 282 335 177 249 202 254 268 185 220 334 300 308 327 329 206 266 217 229 196 207 210 259 283 228 258 322 176 179 197 270 204 215 278 189 290 292 328 175 239 257 277 173 201 291 326 332 235 261 171 186 198 265 285 183 275 339 191 262 304 181 212 195 243 247 250 252 274 294 303 232 317 219 222 227 233 241 331 337 184 199 178 260 299 318 316 336 226 238 281 301 205 211 340 208 218 256 213 302 245 251 203 231 244 325 311 216 293 305 321 271 315 172 193 242] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[74 37 46 100 111 115 167 81 102 112 131 119 21 30 95 107 73 99 132 146 50 85 168 44 61 72 15 39 142 160 5 8 67 109 31 32 77 41 152 47 43 54 68 86 114 155 69 13 2 148 59 169 6 64 159 93 124 135 143 28 56 65 141 45 90 126 58 75 76 103 134 117 156 110 22 57 105 0 52 129 144 11 83 16 138 149 151 137 127 18 104 140 147 157 53 55 62 108 71 89 136 153 97 154 158 162 60 78 118 25 42 113 3 7 106 122 9 23 116 161 170 1 12 17 66 80 84 14 29 48 166 4 34 82 87 139 163 20 26 88 120 96 38 40 24 125 145 10 36 79 121 165 33 130 164 35 92 128 70 91 94 19 49 101 123 133 150 27 51 63 98]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info stats_interval:1000 replication_type:capi worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 active:true]
CheckpointManager 2017-04-25T13:28:43.820-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202...
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:28:43.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
CheckpointManager 2017-04-25T13:28:43.832-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:28:43.837-05:00 [INFO] Checkpointing starts in 1800.568 sec
PipelineRuntimeCtx 2017-04-25T13:28:43.837-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:28:43.837-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:28:43.838-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:28:43.844-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:28:43.844-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:28:43.845-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:28:43.850-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:28:43.851-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:28:43.851-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:28:43.852-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:43.853-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:28:43.853-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:28:43.852-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:28:43.857-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:28:43.860-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:28:43.886-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:28:43.900-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:28:43.900-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:28:43.900-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:28:43.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:28:43.903-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:28:43.903-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:28:43.923-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:28:43.942-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:28:43.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:28:43.958-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:28:43.958-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:28:43.962-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:28:43.962-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:28:43.982-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:28:43.998-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:28:43.998-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:28:43.998-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:28:44.024-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:28:44.024-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:28:44.132-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:28:44.132-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:28:44.132-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:28:44.132-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:28:44.132-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:28:44.132-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:28:44.132-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202 is started----------
PipelineManager 2017-04-25T13:28:44.133-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:28:44.133-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:28:44.134-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:28:44.956-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:28:45.115-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:28:45.121-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:28:45.121-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:28:45.122-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:28:45 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:28:45.300-05:00 [INFO] Send batch count=33
CapiNozzle 2017-04-25T13:28:45.316-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f106/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:28:45.316-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f106/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:28:45.337-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:28:45 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 5 successful
CapiNozzle 2017-04-25T13:28:45.389-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:28:45.389-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:45.390-05:00 [ERROR] Received malformed response from tcp connection vb=2. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:28:45.390-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err Received malformed response from tcp connection vb=2. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:28:45.410-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:28:45.415-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:28:45.429-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:28:45 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 219 successful
CapiNozzle 2017-04-25T13:28:45.444-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:28:45 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 35 successful
CapiNozzle 2017-04-25T13:28:45.468-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:45.469-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:45.489-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:28:45.492-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:28:45.520-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:45.520-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:28:45.520-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:28:45.520-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
ReplicationManager 2017-04-25T13:28:45.525-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
2017/04/25 13:28:45 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 247 successful
PipelineManager 2017-04-25T13:28:45.542-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:28:45.542-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:45.550-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:45.550-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:28:45.550-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:28:45.550-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:45.550-05:00 [INFO] Send batch count=500
2017/04/25 13:28:45 UPR_STREAMREQ for vb 330 successful
PipelineManager 2017-04-25T13:28:45.547-05:00 [INFO] err_list=[{"time":"2017-04-25T13:28:45.547059162-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]
PipelineManager 2017-04-25T13:28:45.557-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:28:45 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 54 successful
CapiNozzle 2017-04-25T13:28:45.655-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:45.656-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:45.665-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:45.666-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:28:45 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 258 successful
CapiNozzle 2017-04-25T13:28:45.675-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:28:45.685-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:28:45 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 282 successful
StatisticsManager 2017-04-25T13:28:45.851-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:28:45.852-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202 {"Errors": "[{\"time\":\"2017-04-25T13:28:45.547059162-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]", "Status": "Pending"}

2017/04/25 13:28:45 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:28:45 UPR_STREAMREQ for vb 99 successful
CheckpointManager 2017-04-25T13:28:46.011-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
CheckpointManager 2017-04-25T13:28:46.011-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:28:46 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:28:46 UPR_STREAMREQ for vb 199 successful
CapiNozzle 2017-04-25T13:28:46.050-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
DcpNozzle 2017-04-25T13:28:46.064-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:28:46.072-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
DcpNozzle 2017-04-25T13:28:46.139-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:28:46.240-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:46.256-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:28:46.423-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:46.434-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:28:46.625-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:46.636-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:28:46.759-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:28:46.759-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:28:46.759-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:28:46.759-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:46.759-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:46.844-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:46.844-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:46.858-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:28:47.037-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:28:47.037-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:28:47.037-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:28:47.037-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:47.037-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:47.159-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:47.160-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:47.177-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:28:47.406-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:28:47.432-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:28:48.252-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:28:48.238-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:48.394-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:28:48.423-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:28:49.447-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:28:49.455-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:28:49.455-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:28:49.457-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:49.528-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:49.702-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:49.786-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:28:49.842-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:49.923-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:49.973-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:28:50.328-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:28:50.328-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:28:50.328-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:28:50.328-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:28:50.329-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:28:50.404-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:28:50.404-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:28:50.426-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:28:50.498-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:28:50.514-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:28:50.942-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:28:50.975-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:28:51.232-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:28:51.254-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:28:53.793-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
GenericSupervisor 2017-04-25T13:28:54.359-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:28:54.253-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:28:55.334-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:28:55.272-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:28:55.781-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries 
GenericSupervisor 2017-04-25T13:28:56.068-05:00 [INFO] ****************************
PipelineManager 2017-04-25T13:28:56.323-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:28:56.328-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:28:56.335-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
GenericSupervisor 2017-04-25T13:28:56.373-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:28:56.654-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
PipelineRuntimeCtx 2017-04-25T13:28:56.532-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:28:56.822-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:28:56.809-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:28:57.557-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:28:57.384-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:28:57.646-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:28:57.761-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:28:57.977-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
CapiNozzle 2017-04-25T13:28:58.307-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
GenericSupervisor 2017-04-25T13:28:58.388-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:58.400-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
CapiNozzle 2017-04-25T13:28:58.541-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:28:58.554-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:28:58.436-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:28:58.502-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:28:58.39530668 -0500 CDT
GenericSupervisor 2017-04-25T13:28:58.654-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:28:58.681-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=3
GenericSupervisor 2017-04-25T13:28:58.494-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:28:58.697-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:28:58.697-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:28:58.712-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-646725202 has been stopped
GenericSupervisor 2017-04-25T13:29:00.022-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:28:59.569-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:29:00.889-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:29:01.871-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
GenericPipeline 2017-04-25T13:29:01.889-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
AsyncListener 2017-04-25T13:29:01.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:02.013-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:02.016-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:02.018-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:02.034-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:29:02.042-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.059-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.061-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.061-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.061-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.061-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.062-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:29:02.062-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
GenericPipeline 2017-04-25T13:29:01.914-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-646725202
AsyncListener 2017-04-25T13:29:02.095-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:01.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericSupervisor 2017-04-25T13:29:02.147-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
AsyncListener 2017-04-25T13:29:02.148-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:02.044-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:29:02.152-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:29:02.162-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:29:02.162-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:02.163-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:02.163-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:02.163-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:29:02.164-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:02.164-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:29:02.164-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:29:02.164-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:29:02.166-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:29:02.171-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:29:02.175-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:29:02.175-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:29:02.175-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:29:02.175-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:29:02.179-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:02.171885969 -0500 CDT
AsyncListener 2017-04-25T13:29:02.087-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:29:02.363-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:02.175866456 -0500 CDT
GenericPipeline 2017-04-25T13:29:03.438-05:00 [INFO] Executing 0x65e930 timed out
CapiNozzle 2017-04-25T13:29:02.881-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:29:02.976-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:29:03.291-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:29:03.820-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:29:03.832-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:29:03.303-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:29:03.840-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:29:03.840-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:29:03.840-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:29:03.427-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:29:03.840-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:29:03.842-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:29:03.842-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:29:03.817-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-646725202 is stopped
PipelineManager 2017-04-25T13:29:03.849-05:00 [ERROR] Received error when stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata - Execution timed out
GenericPipeline 2017-04-25T13:29:03.680-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:29:03.868-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:29:03.868-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
GenericSupervisor 2017-04-25T13:29:03.540-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
2017/04/25 13:29:03 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 175 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 176 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 177 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 178 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 179 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 180 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 181 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 182 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 183 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 184 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 185 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 186 StreamEnd simulated
2017/04/25 13:29:03 Stream Closed for vb 187 StreamEnd simulated
DcpNozzle 2017-04-25T13:29:03.912-05:00 [INFO] Ask uprfeed to close
CapiNozzle 2017-04-25T13:29:03.227-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
GenericSupervisor 2017-04-25T13:29:03.996-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
DcpNozzle 2017-04-25T13:29:04.000-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:29:04.000-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:04.000516522 -0500 CDT
DcpNozzle 2017-04-25T13:29:04.011-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:29:03.931-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:03.921940072 -0500 CDT
GenericSupervisor 2017-04-25T13:29:03.958-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:29:04.029-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
DcpNozzle 2017-04-25T13:29:04.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericSupervisor 2017-04-25T13:29:04.047-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:29:04.160-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:28:45.547059162-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:29:04.478-05:00 [ERROR] Failed to update pipeline f4268e62702130298bf87f17cc481219/metadata/metadata, err=Execution timed out
GenericSupervisor 2017-04-25T13:29:04.912-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:29:05.114-05:00 [ERROR] Update of pipeline f4268e62702130298bf87f17cc481219/metadata/metadata failed with error=Execution timed out
CapiNozzle 2017-04-25T13:29:05.721-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:29:06.024-05:00 [INFO] releasing capi client
GenericSupervisor 2017-04-25T13:29:05.945-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:29:05.834-05:00 [INFO] err_list=[{"time":"2017-04-25T13:29:05.830739097-05:00","errMsg":"Execution timed out"},{"time":"2017-04-25T13:28:45.547059162-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]
PipelineManager 2017-04-25T13:29:06.496-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 1
GenericSupervisor 2017-04-25T13:29:06.604-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:29:06.630-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=Execution timed out 
PipelineManager 2017-04-25T13:29:06.633-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:29:06.637-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:29:06.638-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:29:06.697-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:29:06.881-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:29:06.897-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:29:06.864-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
PipelineManager 2017-04-25T13:29:06.989-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:29:05.830739097-05:00","errMsg":"Execution timed out"},{"time":"2017-04-25T13:28:45.547059162-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:29:07.124-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:29:07.135-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:29:07.135-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:29:07.233-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:29:07.240-05:00 [INFO] Constructed 2 outgoing nozzles
ReplicationManager 2017-04-25T13:29:07.252-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
XDCRRouter 2017-04-25T13:29:07.255-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:29:07.255-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:29:07.255-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:29:07.255-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:29:07.257-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:29:07.263-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:29:07.275-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
GenericSupervisor 2017-04-25T13:29:07.284-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:29:07.285-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
GenericSupervisor 2017-04-25T13:29:07.287-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:07.287-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:07.287-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:07.287-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:07.287-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:29:07.294-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
CheckpointManager 2017-04-25T13:29:07.294-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
CapiUtils 2017-04-25T13:29:07.414-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:29:07.415-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:29:07.417-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:29:07.422-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
StatisticsManager 2017-04-25T13:29:07.439-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:29:07.440-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
XDCRFactory 2017-04-25T13:29:07.442-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:29:07.442-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 is constructed, start it
GenericPipeline 2017-04-25T13:29:07.451-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[32 155 6 27 169 36 139 20 48 150 165 168 30 50 142 148 135 16 84 123 124 67 99 63 151 42 57 117 140 54 88 51 4 10 113 126 3 44 38 41 73 14 19 40 129 161 56 64 108 15 106 65 7 52 97 144 157 107 136 95 141 49 91 85 122 125 154 82 89 24 112 137 5 69 130 131 138 12 121 11 152 47 87 60 31 34 78 162 101 111 80 93 58 143 149 28 35 159 92 158 9 147 83 116 146 164 166 98 128 81 110 120 29 55 119 2 77 100 75 86 1 37 68 94 22 66 114 71 118 134 170 23 46 104 156 132 39 61 127 133 21 103 8 102 26 109 90 13 45 25 70 59 74 79 76 115 0 62 167 72 105 145 153 160 163 18 43 17 53 33 96] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[184 224 265 328 320 236 329 336 175 244 306 185 200 293 333 237 305 299 316 222 246 266 313 252 253 290 310 179 186 255 278 322 198 211 228 239 280 201 202 216 249 292 258 279 300 307 311 331 220 309 177 209 332 298 319 324 223 286 213 318 182 303 229 277 285 196 206 221 283 284 308 217 338 267 315 337 327 176 264 214 218 232 242 276 321 291 295 243 270 272 230 271 274 314 317 235 263 188 257 288 250 268 193 215 294 326 234 296 302 208 225 172 254 330 189 190 199 287 178 210 233 245 261 191 269 281 334 180 171 174 231 251 301 312 187 194 204 207 335 260 282 240 289 192 227 203 226 241 256 339 205 262 340 325 195 173 183 197 247 304 248 273 297 259 181 212 238 275 219 323]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[8 102 26 109 13 45 90 25 70 59 74 79 76 0 62 115 72 105 167 18 43 145 153 160 163 17 53 33 96 6 27 32 155 169 36 139 20 48 30 50 150 165 168 142 148 16 84 135 67 99 123 124 42 57 63 151 54 88 117 140 4 10 51 3 44 113 126 14 19 38 41 73 40 129 56 64 161 15 106 108 7 52 65 97 144 157 107 136 95 141 49 91 85 122 125 154 82 89 24 112 5 69 137 12 121 130 131 138 11 47 87 152 31 34 60 78 162 80 93 101 111 28 35 58 143 149 92 158 159 9 147 83 116 98 128 146 164 166 29 55 81 110 120 2 77 119 100 1 37 75 86 22 66 68 94 114 23 46 71 118 134 170 104 156 39 61 132 21 103 127 133] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[240 289 192 227 203 226 241 256 339 205 262 340 195 325 304 173 183 197 247 248 273 297 259 181 212 238 275 219 323 184 224 265 328 320 236 329 336 175 244 306 185 200 293 333 237 305 299 316 313 222 246 266 290 252 253 310 179 186 255 278 322 198 211 228 239 280 292 201 202 216 249 331 258 279 300 307 311 177 220 309 209 332 223 286 298 319 324 213 318 182 303 229 277 285 196 206 221 283 284 308 217 338 267 315 337 327 176 264 276 214 218 232 242 291 295 321 243 270 272 230 271 274 235 263 314 317 188 257 288 250 268 193 215 294 326 234 296 208 225 302 172 254 330 178 189 190 199 287 261 210 233 245 334 191 269 281 171 180 174 231 251 301 312 207 187 194 204 260 282 335]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[optimistic_replication_threshold:256 log_level:Info stats_interval:1000 active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi filter_expression: worker_batch_size:500]
CheckpointManager 2017-04-25T13:29:07.458-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934...
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:07.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:29:07.476-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:29:07.478-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:29:07.479-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:29:07.481-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:29:07.480-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:29:07.485-05:00 [INFO] Checkpointing starts in 1804.369 sec
PipelineRuntimeCtx 2017-04-25T13:29:07.485-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:29:07.486-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:29:07.488-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:29:07.489-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:29:07.491-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:29:07.491-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:29:07.490-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:29:07.502-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:29:07.575-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:29:07.576-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:29:07.579-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:29:07.597-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:29:07.599-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:29:07.677-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:29:07.767-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:29:07.776-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:29:07.780-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:29:07.901-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:29:07.936-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:29:08.215-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:29:08.299-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:29:08.407-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:29:08.380-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:29:08.381-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:29:08.485-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:29:08.665-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:29:08.667-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:29:08.686-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:29:08.717-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:29:08.717-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:29:08.847-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:29:08.927-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:29:09.107-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:29:09.107-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:29:09.107-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:29:09.107-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:29:09.107-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:29:09.107-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:29:09.116-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 is started----------
PipelineManager 2017-04-25T13:29:09.121-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:29:09.121-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:29:09.123-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:29:09.572-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:29:09.510-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:29:09.553-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:29:09.531-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
2017/04/25 13:29:10 UPR_STREAMREQ for vb 200 successful
StatisticsManager 2017-04-25T13:29:10.215-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:29:10 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:29:10.424-05:00 [INFO] Send batch count=355
2017/04/25 13:29:10 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 206 successful
CapiNozzle 2017-04-25T13:29:10.472-05:00 [INFO] Send batch count=324
2017/04/25 13:29:10 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:29:10 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:29:11 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:29:11 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:29:11 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:29:11 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:29:11 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:29:12 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:29:12 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:29:12 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:29:13 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:29:13 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:29:14 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:29:14 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:29:15.275-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
StatisticsManager 2017-04-25T13:29:16.488-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:29:18.272-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:29:18.587-05:00 [INFO] validateSourceTopology...
PipelineManager 2017-04-25T13:29:15.487-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:29:18.059-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:29:18 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:29:18 UPR_STREAMREQ for vb 106 successful
CapiNozzle 2017-04-25T13:29:18.902-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
StatisticsManager 2017-04-25T13:29:18.950-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:29:18 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 212 successful
StatisticsManager 2017-04-25T13:29:19.041-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:29:05.830739097-05:00\",\"errMsg\":\"Execution timed out\"},{\"time\":\"2017-04-25T13:28:45.547059162-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1269, "dcp_dispatch_time": 1364, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 333, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 14887, "docs_rep_queue": 13571, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 5132, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 333, "set_filtered": 0, "set_received_from_dcp": 14887, "size_rep_queue": 17250848, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 1901}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 148, "docs_opt_repd": 0, "docs_rep_queue": 6095, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 148, "size_rep_queue": 7374966, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 258, "max": 1816, "mean": 1815.515503875969, "min": 1815}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 185, "docs_opt_repd": 0, "docs_rep_queue": 8482, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 185, "size_rep_queue": 10652613, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 292, "max": 1978, "mean": 1976.9486301369864, "min": 1976}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 269, "dcp_dispatch_time": {"count": 6419, "max": 13795, "mean": 34.357, "min": 6}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 9132, "expiry_received_from_dcp": 0, "set_received_from_dcp": 9132}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 8467, "max": 1675447, "mean": 2372.553, "min": 6}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 10631, "expiry_received_from_dcp": 0, "set_received_from_dcp": 10631}}

StatisticsManager 2017-04-25T13:29:19.043-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 9592 items, sent 324 items
StatisticsManager 2017-04-25T13:29:19.043-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 10868 items, sent 355 items
StatisticsManager 2017-04-25T13:29:19.045-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:9592]
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 9592 items, sent 9592 items. streams inactive: [93 41 61 83 137 155 157 165 27 71 85 114 26 30 56 58 136 24 35 66 133 84 86 34 43 44 49 121 146 166 16 69 119 164 15 50 163 92 129 20 46 55 87 63 68 70 76 21 33 48 112 153 156 12 67 81 97 79 132 17 25 37 72 162 32 99 117 158 80 82 134 130 145 11 36 75 89 169 53 154 142 40 98 110 141 95 135 143 42 57 59 78 19 22 131 138 65 73 160 96 113 122 124 38 74 152 126 168 13 39 118 115 144 151 88 91 31 54 60 77 120 127 148 159 128 149 150 18 29 47 94 140 51 62 90 109 139 161 14 45 170 64 125 147 108 111 116 23 28 52 123 167]
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:10868]
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 10869 items, sent 10868 items. streams inactive: [336 175 237 268 287 326 199 251 261 281 312 335 298 311 190 223 239 272 279 181 226 282 333 290 329 187 218 224 254 271 286 184 227 247 258 270 295 182 195 284 294 319 325 225 257 299 231 337 331 196 198 260 273 285 330 176 192 193 276 316 232 338 249 280 317 244 269 314 340 215 234 315 322 332 320 339 183 191 230 256 263 186 197 229 248 262 297 221 323 324 180 212 219 265 292 236 267 277 313 189 293 327 172 178 179 243 250 289 216 233 238 246 275 283 291 188 194 220 228 264 266 171 173 177 222 235 213 245 174 252 318 321 328 185 240 253 255 241 259 274 278 217 334 214 242 288 296]
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =460 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =460 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =238 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =237 
StatisticsManager 2017-04-25T13:29:19.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
2017/04/25 13:29:19 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 17 successful
CapiNozzle 2017-04-25T13:29:19.272-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
GenericSupervisor 2017-04-25T13:29:19.278-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:29:19.298-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:29:19.298-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:29:19.311-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:29:19 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 227 successful
CapiNozzle 2017-04-25T13:29:19.486-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:29:19 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 126 successful
CapiNozzle 2017-04-25T13:29:19.541-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:29:19 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:29:19 UPR_STREAMREQ for vb 128 successful
StatisticsManager 2017-04-25T13:29:19.625-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:29:20 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:29:20 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:29:20 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:29:20 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:29:20 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:29:20 UPR_STREAMREQ for vb 34 successful
CapiNozzle 2017-04-25T13:29:20.098-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:29:22 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:29:23 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:29:23 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:29:23 UPR_STREAMREQ for vb 236 successful
CapiNozzle 2017-04-25T13:29:23.643-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
ToplogyChangeDetector 2017-04-25T13:29:28.258-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:29:27.931-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:29:28.818-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:29:28.305-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:29:28.933-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:29:28.944-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:29:29.096-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:29:31.144-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:29:32.092-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:29:33.229-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
GenericSupervisor 2017-04-25T13:29:33.082-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:33.648-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:29:33 UPR_STREAMREQ for vb 135 successful
GenericSupervisor 2017-04-25T13:29:33.888-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:29:33.787-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
GenericSupervisor 2017-04-25T13:29:35.359-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:29:37.012-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:29:36.212-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:29:37.586-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:29:37.760-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:29:38.287-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:29:40 UPR_STREAMREQ for vb 36 successful
GenericSupervisor 2017-04-25T13:29:40.831-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:29:40.886-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:29:40 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:29:40 UPR_STREAMREQ for vb 237 successful
GenericSupervisor 2017-04-25T13:29:40.685-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:29:40.963-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:29:41.142-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:29:41.170-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:29:05.830739097-05:00\",\"errMsg\":\"Execution timed out\"},{\"time\":\"2017-04-25T13:28:45.547059162-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1851, "dcp_dispatch_time": 2057, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 333, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 38675, "docs_rep_queue": 37910, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 8417, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 333, "set_filtered": 0, "set_received_from_dcp": 38675, "size_rep_queue": 48449521, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 1901}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 148, "docs_opt_repd": 0, "docs_rep_queue": 19840, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 148, "size_rep_queue": 25946640, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 258, "max": 1816, "mean": 1815.515503875969, "min": 1815}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 185, "docs_opt_repd": 0, "docs_rep_queue": 22562, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 185, "size_rep_queue": 29283459, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 292, "max": 1978, "mean": 1976.9486301369864, "min": 1976}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 605, "dcp_dispatch_time": {"count": 17741, "max": 500681, "mean": 797.476, "min": 7}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 20953, "expiry_received_from_dcp": 0, "set_received_from_dcp": 20953}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 254, "dcp_dispatch_time": {"count": 21530, "max": 1489697, "mean": 3096.53, "min": 6}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 23084, "expiry_received_from_dcp": 0, "set_received_from_dcp": 23084}}

StatisticsManager 2017-04-25T13:29:41.172-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 20953 items, sent 324 items
StatisticsManager 2017-04-25T13:29:41.172-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 23845 items, sent 355 items
StatisticsManager 2017-04-25T13:29:41.175-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:20953]
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 20953 items, sent 20952 items. streams inactive: [147 64 52 167 61 83 93 41 71 137 155 157 165 56 58 85 66 136 44 49 84 86 43 69 146 166 163 164 50 55 87 92 46 48 63 68 70 76 153 81 97 156 67 37 72 79 158 162 99 80 82 75 145 169 89 154 53 141 142 40 98 59 78 95 143 42 57 138 73 160 65 96 152 38 74 168 39 151 144 60 77 88 91 54 148 159 47 94 149 150 90 140 51 62 139 161 170 45]
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:23845]
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 23846 items, sent 23845 items. streams inactive: [171 173 177 245 174 252 185 240 253 255 278 241 259 274 242 288 296 336 175 268 287 335 199 251 261 281 272 279 298 190 239 181 282 271 286 290 187 254 258 270 184 247 284 294 295 182 195 257 299 337 285 196 198 260 273 176 192 193 276 338 249 280 340 244 269 256 263 339 183 191 262 297 186 197 248 292 180 265 267 277 189 293 250 172 178 179 243 246 275 289 238 264 266 283 291 188 194]
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =762 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:29:41.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
2017/04/25 13:29:41 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 42 successful
StatisticsManager 2017-04-25T13:29:41.728-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:29:41.734-05:00 [INFO] validateSourceTopology...
PipelineManager 2017-04-25T13:29:41.766-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:29:41 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 141 successful
CapiNozzle 2017-04-25T13:29:41.770-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:29:41 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 340 successful
StatisticsManager 2017-04-25T13:29:41.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:29:41.850-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:29:41.892-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:29:41 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:29:41 UPR_STREAMREQ for vb 243 successful
CapiNozzle 2017-04-25T13:29:42.041-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:29:42 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 249 successful
StatisticsManager 2017-04-25T13:29:42.806-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:29:42 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:29:42 UPR_STREAMREQ for vb 51 successful
CapiNozzle 2017-04-25T13:29:42.930-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:29:42.935-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:29:42.935-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:29:42.936-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:29:42.950-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
ReplicationManager 2017-04-25T13:29:42.942-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
2017/04/25 13:29:46 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:29:46 UPR_STREAMREQ for vb 251 successful
StatisticsManager 2017-04-25T13:29:46.419-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:29:46 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:29:46 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:29:46 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:29:46 UPR_STREAMREQ for vb 152 successful
CapiNozzle 2017-04-25T13:29:46.577-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:29:46.785-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
ToplogyChangeDetector 2017-04-25T13:29:50.364-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:29:50.620-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:29:49.875-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:29:50.627-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
GenericSupervisor 2017-04-25T13:29:52.592-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:29:52.789-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:29:52.790-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:52.582-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:29:52.804-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:29:50.766-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:29:52.937-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:29:52.939-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:29:52.941-05:00 [INFO] Send batch count=500
2017/04/25 13:29:53 UPR_STREAMREQ for vb 253 successful
StatisticsManager 2017-04-25T13:29:53.118-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 859, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 333, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 63620, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 333, "set_filtered": 0, "set_received_from_dcp": 63620, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

DcpNozzle 2017-04-25T13:29:53.461-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:29:53.461-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
PipelineManager 2017-04-25T13:29:53.547-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
2017/04/25 13:29:53 UPR_STREAMREQ for vb 155 successful
PipelineManager 2017-04-25T13:29:53.551-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={Pipeline is running}
]
PipelineManager 2017-04-25T13:29:53.554-05:00 [INFO] err_list=[{"time":"2017-04-25T13:29:53.554571548-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:29:53.572-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:29:53.582-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:29:53.582-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:29:53.582-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:29:53.584-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
PipelineRuntimeCtx 2017-04-25T13:29:53.587-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:29:53.588-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:29:53.603-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
DcpNozzle 2017-04-25T13:29:53.606-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 3
DcpNozzle 2017-04-25T13:29:53.606-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 3
GenericSupervisor 2017-04-25T13:29:53.614-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
CapiNozzle 2017-04-25T13:29:53.623-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:29:53.592-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:53.588853881 -0500 CDT
GenericSupervisor 2017-04-25T13:29:53.639-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:29:53.639-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:29:53.644-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:29:53.644-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:29:53.647-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:29:53.647-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:29:53.648-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:29:53.648-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 has been stopped
GenericPipeline 2017-04-25T13:29:53.649-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-256789934 is stopped
PipelineManager 2017-04-25T13:29:53.649-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:29:53.649-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:29:53.554571548-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:29:53.649-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:29:53.654-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:29:53 UPR_STREAMREQ for vb 254 successful
GenericPipeline 2017-04-25T13:29:53.667-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
GenericPipeline 2017-04-25T13:29:53.668-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
GenericPipeline 2017-04-25T13:29:53.668-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
GenericPipeline 2017-04-25T13:29:53.668-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-256789934
AsyncListener 2017-04-25T13:29:53.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:53.683-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:29:53.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:29:53.685-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:53.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
2017/04/25 13:29:53 UPR_STREAMREQ for vb 53 successful
DcpNozzle 2017-04-25T13:29:53.688-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:29:53.694-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=53
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=54
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=55
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=56
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=57
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=58
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=59
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=60
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=61
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=62
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=63
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=64
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=65
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=66
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=67
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=68
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=69
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=70
DcpNozzle 2017-04-25T13:29:53.696-05:00 [INFO] There is no active stream for vb=71
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=72
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=73
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=74
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=75
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=76
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=77
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=78
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=79
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=80
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=81
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=82
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=83
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=84
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=85
CapiNozzle 2017-04-25T13:29:53.707-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:29:53.707-05:00 [INFO] There is no active stream for vb=86
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=87
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=88
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=89
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=90
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=91
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=92
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=93
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=94
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=95
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=96
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=97
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=98
DcpNozzle 2017-04-25T13:29:53.709-05:00 [INFO] There is no active stream for vb=99
CapiNozzle 2017-04-25T13:29:53.710-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:29:53.710-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:53.710-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:29:53.711-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:29:53.711-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:29:53.711-05:00 [INFO] There is no active stream for vb=171
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=172
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=173
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=174
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=175
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=176
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=177
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=178
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=179
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=180
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=181
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=182
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=183
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=184
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=185
DcpNozzle 2017-04-25T13:29:53.715-05:00 [INFO] There is no active stream for vb=186
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=187
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=188
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=189
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=190
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=191
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=192
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=193
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=194
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=195
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=196
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=197
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=198
DcpNozzle 2017-04-25T13:29:53.717-05:00 [INFO] There is no active stream for vb=199
GenericPipeline 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:29:53.717-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:29:53.717-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.717-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:53.713-05:00 [INFO] There is no active stream for vb=156
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=157
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=158
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=159
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=160
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=161
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=162
2017/04/25 13:29:53 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:29:53 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:29:53 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 4 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 5 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 6 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 7 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 8 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 9 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 10 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 11 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 12 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 13 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 14 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 15 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 16 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 17 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 18 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 19 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 20 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 21 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 22 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 23 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 24 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 25 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 26 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 27 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 28 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 29 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 30 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 31 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 32 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 33 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 34 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 35 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 36 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 37 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 38 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 39 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 40 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 41 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 42 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 43 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 44 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 45 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 46 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 47 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 48 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 49 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 50 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 51 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 52 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 100 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 101 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 102 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 103 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 104 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 105 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 106 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 107 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 108 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 109 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 110 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 111 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 112 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 113 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 114 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 115 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 116 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 117 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 118 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 119 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 120 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 121 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 122 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 123 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 124 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 125 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 126 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 127 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 128 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 129 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 130 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 131 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 132 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 133 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 134 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 135 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 136 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 137 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 138 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 139 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 140 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 141 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 142 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 143 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 144 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 145 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 146 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 147 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 148 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 149 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 150 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 151 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 152 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 153 StreamEnd simulated
2017/04/25 13:29:53 Stream Closed for vb 154 StreamEnd simulated
AsyncListener 2017-04-25T13:29:53.744-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.744-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.744-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.744-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.744-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:29:53.744-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:29:53.744-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:29:53.744-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:29:53.744-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:53.710958357 -0500 CDT
DcpNozzle 2017-04-25T13:29:53.744-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:29:53.744-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:53.717680124 -0500 CDT
CapiNozzle 2017-04-25T13:29:53.744-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:29:53.744-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:29:53.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:29:53.718-05:00 [INFO] There is no active stream for vb=163
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=164
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=165
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=166
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=167
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=168
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=169
DcpNozzle 2017-04-25T13:29:53.745-05:00 [INFO] There is no active stream for vb=170
DcpNozzle 2017-04-25T13:29:53.748-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:29:53.748-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:53.748767679 -0500 CDT
DcpNozzle 2017-04-25T13:29:53.748-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:29:53.748-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:29:53.748-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:29:53 Stream Closed for vb 155 StreamEnd simulated
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=255
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=256
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=257
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=258
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=259
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=260
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=261
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=262
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=263
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=264
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=265
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=266
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=267
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=268
DcpNozzle 2017-04-25T13:29:53.749-05:00 [INFO] There is no active stream for vb=269
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=270
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=271
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=272
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=273
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=274
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=275
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=276
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=277
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=278
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=279
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=280
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=281
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=282
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=283
DcpNozzle 2017-04-25T13:29:53.750-05:00 [INFO] There is no active stream for vb=284
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=285
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=286
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=287
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=288
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=289
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=290
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=291
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=292
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=293
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=294
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=295
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=296
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=297
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=298
DcpNozzle 2017-04-25T13:29:53.761-05:00 [INFO] There is no active stream for vb=299
DcpNozzle 2017-04-25T13:29:53.774-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:29:53.775-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:29:53.774996138 -0500 CDT
DcpNozzle 2017-04-25T13:29:53.775-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:29:53.775-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:29:53.775-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:29:53.856-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:29:53.891-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:29:53.893-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:29:53.893-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:29:53.893-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:29:53.893-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:29:53.893-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:29:53.893-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
AsyncListener 2017-04-25T13:29:54.186-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:29:54.185-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:29:54.188-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:29:54.189-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:29:54.188-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:29:54.189-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
PipelineManager 2017-04-25T13:29:55.045-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:29:53.554571548-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:29:55.992-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:29:55.995-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:29:55.995-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
ReplicationManager 2017-04-25T13:29:56.039-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
XDCRFactory 2017-04-25T13:29:56.057-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:29:56.069-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:29:56.076-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:29:56.076-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:29:56.076-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:29:56.076-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:29:56.077-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:29:56.079-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:29:56.087-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
GenericSupervisor 2017-04-25T13:29:56.097-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:29:56.098-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
GenericSupervisor 2017-04-25T13:29:56.099-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:56.099-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:56.099-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:56.099-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:29:56.099-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:29:56.104-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
CheckpointManager 2017-04-25T13:29:56.104-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
CapiUtils 2017-04-25T13:29:56.141-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:29:56.142-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:29:56.142-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:29:56.145-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
StatisticsManager 2017-04-25T13:29:56.159-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:29:56.160-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
XDCRFactory 2017-04-25T13:29:56.160-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:29:56.160-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123 is constructed, start it
GenericPipeline 2017-04-25T13:29:56.165-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[338 177 299 340 180 228 249 171 201 294 308 316 273 287 288 263 267 315 205 219 278 309 178 186 230 172 195 237 246 301 268 312 272 285 317 337 235 276 298 174 179 181 216 248 282 253 307 325 175 255 290 190 306 289 332 217 215 252 257 296 188 279 295 210 221 241 313 197 212 200 214 231 303 305 194 277 280 321 236 336 225 211 243 274 339 239 291 251 203 242 310 247 293 222 183 329 335 206 208 292 319 185 229 300 318 320 262 198 218 244 265 283 302 220 223 270 324 182 213 224 258 261 269 173 232 266 189 227 304 187 184 259 327 191 209 331 245 256 281 275 176 199 326 234 311 193 328 333 204 240 271 323 254 260 314 192 207 284 233 238 286 297 202 330 196 226 250 264 322 334] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[29 71 80 146 36 52 143 4 153 25 92 11 86 118 135 7 127 155 160 78 16 47 69 10 27 65 95 101 115 159 54 163 74 164 73 108 132 0 121 142 110 33 43 128 9 139 113 57 70 102 8 149 91 104 60 166 61 165 167 32 66 147 158 19 37 42 48 68 82 161 170 17 136 156 157 88 89 100 141 30 51 75 103 140 150 41 45 49 106 6 81 26 24 50 18 56 67 125 168 5 40 138 38 12 64 112 31 124 134 63 99 137 14 55 154 35 96 119 79 87 39 72 98 22 62 97 107 129 15 105 109 133 28 116 162 94 117 1 3 21 59 111 126 2 93 77 90 23 131 44 123 120 122 58 53 76 144 169 20 84 85 145 34 46 114 130 148 151 13 152 83]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[0 108 132 121 110 142 33 9 43 128 113 139 8 57 70 102 91 149 60 104 166 32 61 165 167 19 66 147 158 161 170 17 37 42 48 68 82 88 136 156 157 30 89 100 141 41 51 75 103 140 150 6 45 49 106 26 81 18 24 50 5 56 67 125 168 40 138 12 38 31 64 112 124 134 14 63 99 137 35 55 154 79 96 119 87 22 39 72 98 15 62 97 107 129 28 105 109 133 94 116 162 1 117 2 3 21 59 111 126 93 23 77 90 44 131 123 58 120 122 20 53 76 144 169 34 84 85 145 13 46 114 130 148 151 83 152 29 36 71 80 146 4 52 143 153 11 25 92 7 86 118 135 78 127 155 160 10 16 47 69 27 54 65 95 101 115 159 163 74 73 164] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[253 307 325 175 255 290 190 306 289 217 332 215 252 257 296 188 279 295 210 221 241 313 197 212 231 303 200 214 305 280 321 194 277 225 236 336 274 339 211 243 239 291 251 203 242 310 247 293 222 183 329 335 206 208 292 319 320 185 229 300 318 262 302 198 218 244 265 283 220 223 270 324 182 261 269 213 224 258 173 232 266 187 189 227 304 327 184 259 209 331 191 281 245 256 275 176 199 326 234 311 193 328 333 204 240 271 323 254 260 314 192 207 284 238 286 233 297 202 330 322 334 196 226 250 264 338 340 177 299 171 180 228 249 316 201 294 308 263 273 287 288 267 315 278 309 205 219 230 178 186 237 246 172 195 268 301 312 272 285 317 337 235 276 298 248 282 174 179 181 216]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info replication_type:capi filter_expression: active:true worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 stats_interval:1000]
CheckpointManager 2017-04-25T13:29:56.167-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123...
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:29:56.175-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
CheckpointManager 2017-04-25T13:29:56.177-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:29:56.181-05:00 [INFO] Checkpointing starts in 1803.43 sec
PipelineRuntimeCtx 2017-04-25T13:29:56.181-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:29:56.181-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:29:56.181-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:29:56.184-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:29:56.185-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:29:56.185-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:29:56.185-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:29:56.186-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:29:56.186-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:29:56.184-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:29:56.189-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:29:56.194-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:29:56.231-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:29:56.234-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:29:56.234-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:29:56.236-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:29:56.245-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:29:56.245-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:29:56.248-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:29:56.259-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:29:56.260-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:29:56.274-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:29:56.286-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:29:56.373-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:29:56.376-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:29:56.377-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:29:56.377-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:29:56.387-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:29:56.399-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:29:56.402-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:29:56.426-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:29:56.426-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:29:56.426-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:29:56.443-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:29:56.443-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:29:56.652-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:29:56.663-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:29:56.666-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:29:56.666-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:29:56.677-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:29:56.677-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:29:56.677-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:29:56.677-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:29:56.677-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:29:56.677-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:29:56.678-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123 is started----------
PipelineManager 2017-04-25T13:29:56.678-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:29:56.678-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:29:56.679-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:29:56 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 9 successful
CapiNozzle 2017-04-25T13:29:56.992-05:00 [INFO] Send batch count=11
2017/04/25 13:29:56 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:29:56 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:29:57.003-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:29:57.014-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:29:57 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 117 successful
CapiNozzle 2017-04-25T13:29:57.058-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:29:57.114-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:29:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 221 successful
CapiNozzle 2017-04-25T13:29:57.162-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:29:57 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:29:57.200-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:29:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 338 successful
CapiNozzle 2017-04-25T13:29:57.227-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:29:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 43 successful
CapiNozzle 2017-04-25T13:29:57.263-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:29:57.263-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:29:57.263-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:29:57.263-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:29:57.266-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
StatisticsManager 2017-04-25T13:29:57.274-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
PipelineManager 2017-04-25T13:29:57.286-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:29:57.286-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:29:57.291-05:00 [INFO] err_list=[{"time":"2017-04-25T13:29:57.291138302-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:29:57.291-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:29:57 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 53 successful
CapiNozzle 2017-04-25T13:29:57.392-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:29:57.392-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:29:57 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 254 successful
CapiNozzle 2017-04-25T13:29:57.430-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:29:57.431-05:00 [INFO] Send batch count=303
2017/04/25 13:29:57 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 261 successful
CapiNozzle 2017-04-25T13:29:57.500-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:29:57.501-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:29:57 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 167 successful
CapiNozzle 2017-04-25T13:29:57.512-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:29:57 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 194 successful
CheckpointManager 2017-04-25T13:29:57.699-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
CheckpointManager 2017-04-25T13:29:57.699-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:29:57 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:29:57 UPR_STREAMREQ for vb 299 successful
DcpNozzle 2017-04-25T13:29:57.812-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:29:58.615-05:00 [INFO] Pipeline is no longer running, exit.
DcpNozzle 2017-04-25T13:29:57.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:29:58.989-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:29:58.796-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:29:59.500-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:30:00.247-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:30:02.513-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123 {"Errors": "[{\"time\":\"2017-04-25T13:29:57.291138302-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 38, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 3, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 1869, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 3, "set_filtered": 0, "set_received_from_dcp": 1869, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

PipelineManager 2017-04-25T13:30:03.654-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:30:03.847-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:30:03.853-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:30:03.944-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
PipelineRuntimeCtx 2017-04-25T13:30:03.954-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:30:04.030-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:30:04.046-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:30:04.583-05:00 [INFO] Received finish signal, watch routine exit
CapiNozzle 2017-04-25T13:30:05.103-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
ToplogyChangeDetector 2017-04-25T13:30:05.264-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:30:05.277-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:30:05.171-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:30:05.294-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:30:05.317-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:30:05.322-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:30:05.323-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:30:05.317-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:30:05.326-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:30:05.279518781 -0500 CDT
GenericSupervisor 2017-04-25T13:30:05.356-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:30:05.357-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:30:05.383-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:30:05.383-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:30:05.384-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:30:05.384-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:30:05.392-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-77302123 has been stopped
GenericSupervisor 2017-04-25T13:30:05.512-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:30:05.530-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77302123 is stopped
PipelineManager 2017-04-25T13:30:05.531-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:30:05.533-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:29:57.291138302-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:30:05.540-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:30:05.553-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
GenericPipeline 2017-04-25T13:30:05.557-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
GenericPipeline 2017-04-25T13:30:05.557-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
GenericPipeline 2017-04-25T13:30:05.557-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-77302123
AsyncListener 2017-04-25T13:30:05.558-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:30:05.561-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:30:05.572-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:30:05.577-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
DcpNozzle 2017-04-25T13:30:05.574-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:30:05.584-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:30:05.592-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:30:05.592-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:30:05.594-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:30:05.594-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:30:05.594-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:30:05.594-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:30:05.597-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:30:05.597-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:30:05.597-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:30:05.597-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:30:05.597-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:30:05.597-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:30:05.594551382 -0500 CDT
CapiNozzle 2017-04-25T13:30:05.601-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:30:05.597275399 -0500 CDT
CapiNozzle 2017-04-25T13:30:05.613-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:30:05.613-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:30:05.613-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:30:05.615-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:30:05.615-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:30:05.617-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:30:05.617-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:30:05.619-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
CapiNozzle 2017-04-25T13:30:05.648-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:30:05.648-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:30:05.648-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:30:05.648-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:30:05.677-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:30:05.677-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:30:05.677583463 -0500 CDT
DcpNozzle 2017-04-25T13:30:05.677-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:30:05.677-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:30:05.677-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:30:05.678-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:30:05.678-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:30:05.678574196 -0500 CDT
DcpNozzle 2017-04-25T13:30:05.678-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:30:05.678-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:30:05.678-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:30:05.781-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:30:05.787-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:30:05.787-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:30:05.834-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:30:05.837-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:30:05.840-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:30:05.840-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:30:05.841-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:30:05.841-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:30:05.841-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:30:05.843-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:30:05.848-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
GenericSupervisor 2017-04-25T13:30:05.852-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:30:05.853-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
GenericSupervisor 2017-04-25T13:30:05.853-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:05.853-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:05.853-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:05.853-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:05.853-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:30:05.857-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
CheckpointManager 2017-04-25T13:30:05.857-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
CapiUtils 2017-04-25T13:30:05.882-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:30:05.882-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:30:05.883-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:30:05.886-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
StatisticsManager 2017-04-25T13:30:05.895-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:30:05.896-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
XDCRFactory 2017-04-25T13:30:05.896-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:30:05.897-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604 is constructed, start it
GenericPipeline 2017-04-25T13:30:05.898-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[212 214 231 321 284 213 259 249 177 228 206 209 229 236 251 281 178 338 184 282 300 215 219 322 334 174 190 208 217 266 310 254 234 294 330 257 261 187 263 172 182 235 286 244 271 173 220 240 279 224 203 316 227 238 307 181 308 296 180 268 312 329 333 302 331 201 309 171 195 248 260 262 313 191 267 247 276 295 317 185 207 222 239 297 299 233 289 301 183 193 336 223 194 288 325 246 293 258 211 256 278 318 324 337 255 304 175 230 269 176 192 221 232 245 199 225 197 196 198 200 287 305 306 179 188 205 216 241 270 189 265 274 292 315 264 277 290 218 250 242 311 314 339 319 335 272 273 298 202 275 291 303 186 323 243 326 328 280 285 327 210 237 332 320 252 253 204 226 283 340] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[66 89 93 30 67 128 47 85 163 7 57 59 108 23 36 88 114 119 126 21 39 115 44 109 106 129 159 62 90 160 37 61 60 139 14 31 34 144 8 110 38 102 73 80 124 6 50 91 158 19 97 72 148 40 16 25 46 133 78 117 15 84 130 68 10 28 101 146 156 5 83 13 164 105 111 27 63 54 22 29 94 33 140 141 116 168 64 65 87 121 162 169 53 70 75 76 118 165 24 55 2 96 100 107 56 69 152 20 58 17 145 103 142 51 98 112 131 32 52 74 127 9 26 71 137 132 135 167 81 125 122 0 11 138 113 136 82 104 95 120 170 77 86 157 99 154 153 123 151 166 12 43 155 161 92 143 35 49 79 147 3 45 149 1 41 48 134 4 42 18 150]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[90 106 129 159 62 61 160 37 31 60 139 14 34 110 144 8 102 38 50 73 80 124 6 158 91 97 19 148 72 25 40 16 133 46 78 84 117 15 130 28 68 10 83 101 146 156 5 164 13 63 105 111 27 29 54 22 94 140 33 141 65 116 168 64 70 87 121 162 169 53 55 75 76 118 165 24 2 69 96 100 107 56 58 152 20 145 17 103 98 142 51 52 112 131 32 26 74 127 9 137 71 125 132 135 167 81 11 122 0 138 104 113 136 82 86 95 120 170 77 154 157 99 151 153 123 43 166 12 143 155 161 92 49 35 45 79 147 3 41 149 1 42 48 134 4 150 18 89 66 67 93 30 85 128 47 57 163 7 36 59 108 23 39 88 114 119 126 21 109 115 44] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[219 322 334 174 190 208 217 266 310 254 234 294 330 257 261 263 187 172 182 235 271 286 244 173 220 240 279 224 203 316 227 238 307 181 296 308 180 268 312 329 333 331 302 171 201 309 195 248 260 262 313 191 267 247 276 295 317 185 207 222 239 297 299 183 233 289 301 193 336 194 223 288 325 246 293 258 211 255 256 278 318 324 337 304 175 230 269 176 192 221 232 245 197 199 225 196 198 200 287 305 306 179 188 205 216 241 270 189 265 274 292 315 218 264 277 290 250 242 311 314 335 339 319 272 273 298 202 275 291 303 186 323 243 326 328 280 285 327 210 237 332 253 320 252 204 226 283 340 212 214 231 321 259 284 213 228 249 177 206 209 229 236 251 281 178 338 282 184 300 215]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 target_nozzle_per_node:2 log_level:Info active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:30:05.901-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604...
AsyncListener 2017-04-25T13:30:05.904-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:05.905-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
StatisticsManager 2017-04-25T13:30:05.905-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:30:05.916-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:30:05.919-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:30:05.919-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:30:05.920-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
StatisticsManager 2017-04-25T13:30:05.916-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:30:05.921-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:30:05.921-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:30:05.922-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:30:05.926-05:00 [INFO] Checkpointing starts in 1801.201 sec
PipelineRuntimeCtx 2017-04-25T13:30:05.926-05:00 [INFO] Service CheckpointManager has been started
CapiUtils 2017-04-25T13:30:05.930-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:30:05.930-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
GenericSupervisor 2017-04-25T13:30:05.931-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:30:05.931-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:30:05.932-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:30:05.934-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:30:05.953-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:30:05.953-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:30:05.953-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:30:05.973-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:30:05.973-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:30:05.990-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:30:05.998-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:30:06.009-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:30:06.050-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:30:06.050-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:30:06.050-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:30:06.074-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:30:06.085-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:30:06.085-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:30:06.109-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:30:06.109-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:30:06.109-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:30:06.124-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:30:06.124-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:30:06.138-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:30:06.261-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:30:06.262-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:30:06.262-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:30:06.262-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:30:06.262-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:30:06.263-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:30:06.263-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604 is started----------
PipelineManager 2017-04-25T13:30:06.263-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:30:06.263-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:30:06.267-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:30:06.344-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:30:06.351-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:30:06.352-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:30:06.358-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:30:06 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:30:06.543-05:00 [INFO] Send batch count=500
2017/04/25 13:30:06 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 208 successful
CapiNozzle 2017-04-25T13:30:06.672-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:06.676-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:06.690-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:30:06 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 113 successful
CapiNozzle 2017-04-25T13:30:06.725-05:00 [INFO] Send batch count=176
2017/04/25 13:30:06 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:30:06 UPR_STREAMREQ for vb 321 successful
PipelineManager 2017-04-25T13:30:07.467-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
2017/04/25 13:30:08 UPR_STREAMREQ for vb 23 successful
CapiNozzle 2017-04-25T13:30:08.319-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:30:08.405-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
PipelineManager 2017-04-25T13:30:08.283-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:30:09.230-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:30:27.767-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:30:27.784-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:30:27.661-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604
CapiNozzle 2017-04-25T13:30:27.859-05:00 [ERROR] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 - Connection reset failed
CapiNozzle 2017-04-25T13:30:27.861-05:00 [ERROR] Raise error condition dial tcp 10.202.98.33:9091: i/o timeout
2017/04/25 13:30:27 UPR_STREAMREQ for vb 225 successful
GenericSupervisor 2017-04-25T13:30:27.690-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:30:27.907-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:30:27.945-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:30:27.950-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:30:27.953-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:30:27 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 127 successful
PipelineRuntimeCtx 2017-04-25T13:30:28.027-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:30:28.033-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:30:28.034-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:30:28.036-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:30:28.079-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:30:28.120-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
GenericSupervisor 2017-04-25T13:30:27.871-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:dial tcp 10.202.98.33:9091: i/o timeout], but error is ignored. pipeline_state=3
StatisticsManager 2017-04-25T13:30:28.153-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 376, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 7293, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 376, "set_filtered": 0, "set_received_from_dcp": 7293, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

StatisticsManager 2017-04-25T13:30:28.188-05:00 [INFO] StatisticsManager Stopped
CapiNozzle 2017-04-25T13:30:28.267-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:30:28 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 226 successful
GenericSupervisor 2017-04-25T13:30:28.781-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:30:28.784-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:30:28.784-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:30:28 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:30:28 UPR_STREAMREQ for vb 30 successful
ToplogyChangeDetector 2017-04-25T13:30:28.865-05:00 [INFO] Received finish signal, watch routine exit
ToplogyChangeDetector 2017-04-25T13:30:28.869-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:30:28.870-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:30:28.877-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:30:28.878-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:30:28.879-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:30:28.87718273 -0500 CDT
GenericSupervisor 2017-04-25T13:30:28.886-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:30:28.886-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:30:28.886-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-841116604 has been stopped
GenericPipeline 2017-04-25T13:30:28.888-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-841116604 is stopped
PipelineManager 2017-04-25T13:30:28.888-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:30:28.892-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={Pipeline is stopped}

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x6b324d]

goroutine 5775 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).tcpProxy(0xc42bfe1680, 0x64, 0xc269898b40, 0xc328f5f650, 0xc269898a80, 0xc269898ae0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:908 +0x7cd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:831 +0xc91

goroutine 1 [chan receive, 6 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 6 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 6 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc20802b9b0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080d6001, 0x5ff, 0x5ff, 0xc20800bd00, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080d6001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080d2000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d2000, 0x8f07a0, 0xc2080664a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc208066480, 0xc20801fc60, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bc00, 0x7efee181ac10, 0xc208066480, 0x0, 0x0, 0xc20801fc60, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bc00, 0x7efee181ac10, 0xc208066480, 0xc208086280, 0xc20801f660, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bc00, 0x7efee181ac10, 0xc208066480)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b9f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b9f90, 0xc2080be480, 0x7efee1819398, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 19 [syscall, 6 minutes]:
syscall.Syscall(0x0, 0x0, 0xc2080e5000, 0x1000, 0xc20803b270, 0xc20814e1e0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc2080e5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc2080e5000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc2080e5000, 0x1000, 0x1000, 0xc20803b270, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc2080e5000, 0x1000, 0x1000, 0xc20807b360, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208066f60)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208066f60, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 12 [select, 6 minutes]:
net/http.(*persistConn).readLoop(0xc2080774a0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 13 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc2080774a0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4927 [chan send]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).UpdateSettings(0xc3f9e52a80, 0xc2c57476b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:628 +0x2b7
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).UpdateSettings(0xc3f9ef4790, 0xc2c57476b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:633 +0x2b0
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3f9d15950, 0x100, 0xc24f5bdc50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:392 +0x9b8
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3f9d15950, 0x2, 0xc3fe5fe450, 0x64, 0x98, 0xc2c55cfa70, 0x0, 0xc2c51ab590, 0xc2b6f92e60, 0xc40320c9c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 20 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc2080821c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 21 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080821d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 22 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 23 [select, 6 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 24 [select, 6 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080ec7b0, 0xf, 0xc208144fc0, 0xc208066900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080ec7b0, 0xf, 0xc208144fc0, 0xc208066900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080ec7b0, 0xf, 0xc208144fc0, 0xc208066900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc20807aeb0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 25 [select, 6 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080fe700, 0x11, 0xc208143fc0, 0xc208066900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080fe700, 0x11, 0xc208143fc0, 0xc208066900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080fe700, 0x11, 0xc208143fc0, 0xc208066900, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc20807ad20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 26 [select]:
net/http.(*persistConn).roundTrip(0xc33a917600, 0xc21e6b39f0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc2583d0270, 0xc2a6cbc720, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc2583d0270, 0x7efee1818fe8, 0xe94600, 0x54, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc2583d0270, 0x54, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc2583d0270, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc2583d0270, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc2583d0270, 0x7efee181af78, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc326d16d90, 0xc23ffa54c0, 0x3c, 0x7efee181af78, 0xc2408295e0, 0x80b080, 0xc24c4d6700, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc3e2388bf8, 0xc23ffa54c0, 0x3c, 0x80b080, 0xc24c4d6700, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Pool).refresh(0xc3e2388bd0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:740 +0x1a2
github.com/couchbase/go-couchbase.(*Client).GetPool(0xc2c2096c60, 0x9cae60, 0x7, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:771 +0x396
github.com/couchbase/goxdcr/utils.RemotePool(0xc2080de140, 0x11, 0xc20802bd40, 0xd, 0xc20802bd50, 0xc, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:98 +0x4c5
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).cacheRef(0xc208066360, 0xc4364e35f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:512 +0x1d3
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).refresh(0xc208066360, 0xc4364e35f0, 0xc2080fe280, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:551 +0x154
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).RemoteClusterByUuid(0xc208066360, 0xc2080fe280, 0x20, 0x1, 0xc23fd49dc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:178 +0x2a1
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc208010460, 0xc4365d4100, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:480 +0x51d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc208010460, 0xc4365d4100)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc208104540)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 27 [select]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208066f00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:102 +0x75b
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 5697 [select]:
net/http.(*persistConn).roundTrip(0xc3e1e1f8c0, 0xc232391e70, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc258b8e820, 0xc25b2eecc0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc258b8e820, 0x7efee1818fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc258b8e820, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc258b8e820, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc258b8e820, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc258b8e820, 0x0, 0x818220, 0xc435e8fe10, 0xeb0d80, 0xc2080ec610, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc42c1e3d77, 0x11, 0x9ed550, 0xe, 0x0, 0xc20802bd40, 0xd, 0xc20802bd50, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080ec620, 0x9ed550, 0xe, 0xc2835e4b40, 0xeb0d80, 0x5, 0x66504c, 0x30000c435e8fe08, 0xc42c465040, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080ec620, 0xc42c0c9260, 0xc24c38af20, 0xc3bd8f5e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc42c3824b0, 0x2, 0xc42c4e9210, 0x64, 0x98, 0xc42c1e3e90, 0xc3d4005f00, 0xc434d9e450, 0xc4351b77c0, 0xc4351b80c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 29 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208066f18)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 30 [IO wait]:
net.(*pollDesc).Wait(0xc208104a70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208104a70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208104a10, 0x0, 0x7efee1818df0, 0xc2143cf4f0)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc2081020a8, 0xc208146e38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc2081020a8, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208067320, 0x7efee181b6a8, 0xc2081020a8, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 31 [select, 6 minutes]:
net/http.(*persistConn).readLoop(0xc20808a160)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 32 [select, 6 minutes]:
net/http.(*persistConn).writeLoop(0xc20808a160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 33 [IO wait, 6 minutes]:
net.(*pollDesc).Wait(0xc2080115d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080115d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011570, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc20810cc88)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038178, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc208038178, 0xc2080774f8, 0xc2080c5000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801ff40, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208066840)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208066840, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208066840, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf0b0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf0b0, 0xc2080d7200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc20800ba40, 0xc2080d7200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc20800ba40, 0xc2080d7200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc20800ba80, 0xc2080d7200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d2300, 0xc2080ea8c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d2300, 0x80f160, 0xc20800bac0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 35 [IO wait, 6 minutes]:
net.(*pollDesc).Wait(0xc208104b50, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208104b50, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208104af0, 0xc208160000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc20810ce30)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081020b0, 0xc208160000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc2081020b0, 0xc20808a1b8, 0xc208160000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080feb80, 0xc208160000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208067500)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208067500, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208067500, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf230)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf230, 0xc2080d7800, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc20800bbc0, 0xc2080d7800, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc20800bbc0, 0xc2080d7800, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc20800bd00, 0xc2080d7800, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d2600, 0xc2080eaac0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d2600, 0x80f160, 0xc20800be00, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 2467 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0xc9, 0xc305a74a00, 0xec, 0x100, 0xc2dbc65800, 0xbd, 0x100, 0xc2eceb50e0, 0xc2eceb5020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 532 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x64, 0xc234fe6700, 0xec, 0x100, 0xc23b276c00, 0x56, 0x80, 0xc2410ac300, 0xc2410ac240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 60 [IO wait]:
net.(*pollDesc).Wait(0xc208011560, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011560, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011500, 0xc2081a8000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc434bf5448)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038100, 0xc2081a8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc208038100, 0xc20808a268, 0xc2081a8000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080df5c0, 0xc2081a8000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20811e5a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20811e5a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 46 [IO wait]:
net.(*pollDesc).Wait(0xc208105090, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208105090, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208105030, 0xc208106000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc40fec9448)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081021b0, 0xc208106000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc2081021b0, 0xc20808a5d8, 0xc208106000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080dfce0, 0xc208106000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20811f020)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20811f020, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a580)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 47 [select]:
net/http.(*persistConn).writeLoop(0xc20808a580)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 61 [select]:
net/http.(*persistConn).writeLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 331 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x1, 0xc221a79200, 0xea, 0x100, 0xc2218c3800, 0xc7, 0x100, 0xc221f715c0, 0xc221f71500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1407 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d440, 0x67, 0xc2d225b400, 0xec, 0x100, 0xc2d1b61000, 0xb3, 0x100, 0xc2d261d680, 0xc2d261d5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1707 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0x12d, 0xc2dc869a00, 0xec, 0x100, 0xc2eba5d800, 0xcf, 0x100, 0xc2ebf313e0, 0xc2ebf31320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2026 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48240, 0x0, 0xc333cbb200, 0xea, 0x100, 0xc333a97800, 0xb8, 0x100, 0xc333f316e0, 0xc333f31620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3065 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38aeff440, 0x12d, 0xc395d06600, 0xec, 0x100, 0xc3959e5800, 0xcc, 0x100, 0xc3960343c0, 0xc396034300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2717 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38fc0, 0x12d, 0xc23fc3c500, 0xec, 0x100, 0xc3725ce000, 0xcf, 0x100, 0xc3726d4f00, 0xc3726d4e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2454 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0x12d, 0xc34ff93f00, 0xec, 0x100, 0xc3452e0800, 0xcf, 0x100, 0xc25c9dc660, 0xc25c9dc5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1365 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d440, 0x64, 0xc2b2ecc500, 0xec, 0x100, 0xc2b2899800, 0xc9, 0x100, 0xc2b30c4c00, 0xc2b30c4b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2050 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48240, 0x64, 0xc337d0f200, 0xec, 0x100, 0xc3418e7800, 0xc9, 0x100, 0xc341a9d1a0, 0xc341a9d0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5713 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc42bfe1680, 0x64, 0xc435902b00, 0xec, 0x100, 0xc4355b2800, 0xc9, 0x100, 0xc435b4a8a0, 0xc435b4a7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 987 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x1, 0xc221a79c00, 0xea, 0x100, 0xc28528e000, 0xc7, 0x100, 0xc28775f680, 0xc28775f5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1024 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0x12f, 0xc24157cb00, 0xec, 0x100, 0xc29a48e800, 0xce, 0x100, 0xc29a93b380, 0xc29a93b2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2484 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0x12f, 0xc2dc868e00, 0xec, 0x100, 0xc2ede51000, 0xce, 0x100, 0xc31cf8cc60, 0xc31cf8cba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1287 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d440, 0x0, 0xc2a6ee1900, 0xea, 0x100, 0xc2a709c000, 0xb8, 0x100, 0xc2a70c7560, 0xc2a70c74a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4004 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c20d78c0, 0x4, 0xc24f5ee000, 0xea, 0x100, 0xc3724b8800, 0xa2, 0x100, 0xc371a05d40, 0xc371a05c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 636 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x0, 0xc24f5ee600, 0xea, 0x100, 0xc24ee81000, 0xb8, 0x100, 0xc24f6872c0, 0xc24f687200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5791 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d68d1d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2442 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0xc8, 0xc312390400, 0xec, 0x100, 0xc33bc21800, 0xbb, 0x100, 0xc2a6f720c0, 0xc2a6f72000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 966 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x6b, 0xc292ac6300, 0xec, 0x100, 0xc292ba8000, 0xbd, 0x100, 0xc292c7e060, 0xc292b3bf80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1646 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0xc9, 0xc23fc3cb00, 0xec, 0x100, 0xc2d941e800, 0xa2, 0x100, 0xc255282e40, 0xc255282d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 924 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x0, 0xc27ca32300, 0xea, 0x100, 0xc27bf93000, 0xb8, 0x100, 0xc27caf1620, 0xc27caf1560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 928 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0xc8, 0xc27d2fb400, 0xec, 0x100, 0xc27cc12800, 0xbb, 0x100, 0xc27d66f320, 0xc27d66f260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5633 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d71c0f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 970 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0x130, 0xc295482100, 0xec, 0x100, 0xc295117000, 0xa3, 0x100, 0xc2955918c0, 0xc295591800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2421 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0x12c, 0xc336cb5e00, 0xec, 0x100, 0xc354fc3000, 0xab, 0x100, 0xc3580a2720, 0xc3580a2660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4925 [semacquire]:
sync.(*Mutex).Lock(0xc3fa316af8)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3f9d15950, 0x39, 0xc24e3186c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3f9d15950, 0x0, 0xc3fe5fe2c0, 0x64, 0x160, 0xc2c55cfa70, 0x0, 0xc2c51ab590, 0xc2b6f92e60, 0xc40320c9c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 879 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0x130, 0xc208163400, 0xec, 0x100, 0xc2757dd200, 0x28, 0x40, 0xc2758be240, 0xc2758be180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2437 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0x130, 0xc356a13f00, 0xec, 0x100, 0xc34e5ab000, 0xbc, 0x100, 0xc2c73f7560, 0xc2c73f74a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5772 [runnable]:
net.(*pollDesc).Wait(0xc32834f480, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc32834f480, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc32834f420, 0xc24c4d61e0, 0x18, 0x18, 0x0, 0x7efee1818df0, 0xc21e6b3938)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080387e8, 0xc24c4d61e0, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7efee181ab08, 0xc2080387e8, 0xc24c4d61e0, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7efee181ab08, 0xc2080387e8, 0xc24c4d61e0, 0x18, 0x18, 0xc4373a2000, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc2a6cbc660, 0x7efee181ab08, 0xc2080387e8, 0xc24c4d61e0, 0x18, 0x18, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:154 +0x11e
github.com/couchbase/gomemcached/client.getResponse(0x7efee181ab08, 0xc2080387e8, 0xc24c4d61e0, 0x18, 0x18, 0xc2a6cbc660, 0x60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Send(0xc25b97c750, 0xc2a6cbc600, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:80 +0x10d
github.com/couchbase/gomemcached/client.(*Client).AuthList(0xc25b97c750, 0xc20801f940, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:123 +0x67
github.com/couchbase/gomemcached/client.(*Client).Auth(0xc25b97c750, 0xc20802b890, 0x6, 0xc20801f960, 0x20, 0xc20801f960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:128 +0x4b
github.com/couchbase/cbauth.func·005(0x7efee1819448, 0xc2080380e8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/mcdauthhandler.go:66 +0xf8
github.com/couchbase/cbauth.WithAuthenticator(0x7efee1819448, 0xc2080380e8, 0xc2080b5a70, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:123 +0x8a
github.com/couchbase/cbauth.(*AuthHandler).AuthenticateMemcachedConn(0xc24af3fc60, 0xc241072c00, 0x1d, 0xc25b97c750, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/mcdauthhandler.go:71 +0x77
github.com/couchbase/go-couchbase.defaultMkConn(0xc241072c00, 0x1d, 0x7efee181ba78, 0xc24af3fc60, 0x7efed8ccda00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:64 +0x162
github.com/couchbase/go-couchbase.(*connectionPool).GetWithTimeout(0xc35885dcc0, 0x9356907420000, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:152 +0x635
github.com/couchbase/go-couchbase.(*connectionPool).Get(0xc35885dcc0, 0xc2337f30f8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:165 +0x46
github.com/couchbase/go-couchbase.getStatsParallel(0xc23dcde340, 0x1, 0x9cac80, 0x3, 0xc2aa837d40)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:135 +0x10d
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 4770 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc368ca6240, 0x12f, 0xc240ac5f00, 0xec, 0x100, 0xc3bfc4d000, 0xce, 0x100, 0xc3bfe194a0, 0xc3bfe193e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5513 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc41cb74b40, 0x0, 0xc4265b9700, 0xea, 0x100, 0xc426db2000, 0xb3, 0x100, 0xc426e55200, 0xc426e55140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 952 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x64, 0xc284dfd700, 0xec, 0x100, 0xc287957800, 0xac, 0x100, 0xc2880ee9c0, 0xc2880ee900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5223 [select]:
net/http.(*persistConn).writeLoop(0xc40f862630)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3750 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c20d78c0, 0x67, 0xc3d4166900, 0xec, 0x100, 0xc3d4045800, 0xb3, 0x100, 0xc3d4312300, 0xc3d4312240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1737 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0x12c, 0xc2f2715600, 0xec, 0x100, 0xc2f6e7f000, 0xc4, 0x100, 0xc2f73e3da0, 0xc2f73e3ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2415 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f1b8cb40, 0xcc, 0xc356a12a00, 0xec, 0x100, 0xc2b795d800, 0xb7, 0x100, 0xc3004c2c00, 0xc3004c2b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1689 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0xca, 0xc23dc1d400, 0xec, 0x100, 0xc2bf521000, 0xbc, 0x100, 0xc2b9cad740, 0xc2b9cad680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 80 [IO wait]:
net.(*pollDesc).Wait(0xc20832bfe0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20832bfe0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20832bf80, 0xc20c6ae000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc2113f7198)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081021f0, 0xc20c6ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc2081021f0, 0xc208076688, 0xc20c6ae000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c6a4660, 0xc20c6ae000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20832cde0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20832cde0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076630)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 113 [select]:
net/http.(*persistConn).writeLoop(0xc208076630)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 116 [IO wait]:
net.(*pollDesc).Wait(0xc20c6b07d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c6b07d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c6b0770, 0xc20c6c8000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc22c7223c8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208102288, 0xc20c6c8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc208102288, 0xc2080767e8, 0xc20c6c8000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c6bf780, 0xc20c6c8000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20832d740)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20832d740, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076790)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 117 [select]:
net/http.(*persistConn).writeLoop(0xc208076790)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 145 [syscall, 6 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 683 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x68, 0xc23dc1c300, 0xec, 0x100, 0xc269bf9800, 0xc2, 0x100, 0xc26a020fc0, 0xc26a020f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 345 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x64, 0xc20802f800, 0xec, 0x100, 0xc2250e8000, 0xc9, 0x100, 0xc22514d920, 0xc22514d860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 360 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xcb, 0xc22dd69200, 0xec, 0x100, 0xc22d9f7800, 0xb6, 0x100, 0xc22e0cf920, 0xc22e0cf860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3342 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9d40, 0x12c, 0xc3b2307300, 0xec, 0x100, 0xc3b2e0d000, 0xc4, 0x100, 0xc3b39e2420, 0xc3b39e2360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 838 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x64, 0xc210eff300, 0xec, 0x100, 0xc2752e8600, 0x3b, 0x40, 0xc2753eade0, 0xc2753ead20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3448 [semacquire, 2 minutes]:
sync.(*WaitGroup).Wait(0xc3cb46c3c0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc3c1ef34a0, 0xc2080ea380, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc20814e140, 0x7efee18228b0, 0xc3c223cb00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7efee18228b0, 0xc3c223cb00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 5762 [IO wait]:
net.(*pollDesc).Wait(0xc3c68048b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3c68048b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3c6804850, 0xc3cb984000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc2337f2c28)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc43651c028, 0xc3cb984000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc4364f80e8, 0xc3cb984000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc3ddd28f40, 0xc3cb984000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc35880a060)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc35880a060, 0xc23ffa5c0a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc35880a060, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc24132fd10, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc24132fd10, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc35880a060, 0xc24b8c0f70, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc4364f80a0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc4364f80a0)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 661 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x6e, 0xc24f5eeb00, 0xec, 0x100, 0xc264169000, 0x1f4, 0x200, 0xc23271b680, 0xc23271b5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4082 [runnable]:
net.(*pollDesc).Wait(0xc367cde1b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc367cde1b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc367cde150, 0xc317029000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc21e6b39e8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc29be2f510, 0xc317029000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc29be2f510, 0xc33a917658, 0xc317029000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc31528d300, 0xc317029000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc300928000)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc300928000, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc33a917600)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 333 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12e, 0xc210effc00, 0xec, 0x100, 0xc223c62000, 0xb4, 0x100, 0xc21f6a6c60, 0xc21f6a6ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 692 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x65, 0xc234fe6e00, 0xec, 0x100, 0xc26a9f9000, 0xc6, 0x100, 0xc26ad56ae0, 0xc26ad56a20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1990 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48240, 0x1, 0xc3244fef00, 0xea, 0x100, 0xc324467800, 0xc7, 0x100, 0xc324ae87e0, 0xc324ae8720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2733 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38fc0, 0xc8, 0xc2f2715e00, 0xec, 0x100, 0xc385a28800, 0xbb, 0x100, 0xc385fe2480, 0xc385fe23c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 374 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12f, 0xc22dd69700, 0xec, 0x100, 0xc22d416800, 0xce, 0x100, 0xc22570e5a0, 0xc22570e4e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1573 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x0, 0xc2a66c8000, 0xea, 0x100, 0xc2c0b7b000, 0x45, 0x80, 0xc2da9d2a20, 0xc2da9d2960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 302 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x0, 0xc20802eb00, 0xea, 0x100, 0xc21a974000, 0xb8, 0x100, 0xc21aa46300, 0xc21aa46240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 294 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x130, 0xc208162f00, 0xec, 0x100, 0xc21561c000, 0xbc, 0x100, 0xc2156baf00, 0xc2156bae40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4395 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea6c0, 0xc9, 0xc395d06100, 0xec, 0x100, 0xc3d74bb800, 0xbd, 0x100, 0xc385a75440, 0xc385a75380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 665 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x2, 0xc240ac5900, 0xea, 0x100, 0xc26639f800, 0xcf, 0x100, 0xc26781c360, 0xc26781c2a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5691 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc42c0da000, 0xc434b3fcb0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 5631 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d68dcc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 1755 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x2, 0xc305549000, 0xea, 0x100, 0xc305357000, 0xcf, 0x100, 0xc305783740, 0xc305783680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1992 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48480, 0xc9, 0xc3273f0500, 0xec, 0x100, 0xc326f57000, 0xbd, 0x100, 0xc327429a40, 0xc327429980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3396 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9b00, 0x2, 0xc3c00eb800, 0xea, 0x100, 0xc3c0df7000, 0xcf, 0x100, 0xc3c1742f60, 0xc3c1742ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5723 [IO wait]:
net.(*pollDesc).Wait(0xc4375e3e20, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc4375e3e20, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc4375e3dc0, 0xc324050000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc2260c13a0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc435e8fe18, 0xc324050000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc435ee78a8, 0xc324050000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc4377c25e0, 0xc324050000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc358faa060)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc358faa060, 0xc4373a210a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc358faa060, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc241322c90, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc241322c90, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc358faa060, 0xc25561bee0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc435ee7860, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc435ee7860)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 1383 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d440, 0x4, 0xc2b9e61700, 0xea, 0x100, 0xc2c2a19800, 0xa2, 0x100, 0xc2c2e62ba0, 0xc2c2e62ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 648 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x64, 0xc258b2e900, 0xec, 0x100, 0xc25c2c8800, 0xa9, 0x100, 0xc25ca92240, 0xc25ca92180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 313 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xc9, 0xc215d15100, 0xec, 0x100, 0xc21cb7e000, 0xbd, 0x100, 0xc21cbce900, 0xc21cbce840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 715 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411beb40, 0xcc, 0xc22dd69400, 0xec, 0x100, 0xc26bc68000, 0xb7, 0x100, 0xc26bcac8a0, 0xc26bcac7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3518 [semacquire]:
sync.(*Mutex).Lock(0xc3c236c5e0)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3c1ef34a0, 0x50, 0xc2bd683500, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3c1ef34a0, 0x0, 0xc3c24cc580, 0x64, 0x160, 0xc3c240ef90, 0x40ec00, 0xc3c700e3c0, 0xc3cb46c3c0, 0xc3cb4686c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 5632 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d726460)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 991 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0x12c, 0xc28a481a00, 0xec, 0x100, 0xc29396f800, 0xc4, 0x100, 0xc2946a4960, 0xc2946a48a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 366 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x2, 0xc215d14800, 0xea, 0x100, 0xc221419000, 0xcf, 0x100, 0xc221f71740, 0xc221f71680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5628 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d70b590)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4344 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea480, 0x0, 0xc3c0324400, 0xea, 0x100, 0xc3d6f6b000, 0xa2, 0x100, 0xc3e0667560, 0xc3e06674a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 393 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12c, 0xc23dc1c100, 0xec, 0x100, 0xc23d60b000, 0xc4, 0x100, 0xc23dba98c0, 0xc23dba9800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 406 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x4, 0xc23e4f2200, 0xea, 0x100, 0xc23d693000, 0xa2, 0x100, 0xc23e5ecd80, 0xc23e5eccc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 384 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x65, 0xc234fe7300, 0xec, 0x100, 0xc237b7f000, 0x97, 0x100, 0xc237fc9c80, 0xc237fc9bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3519 [semacquire]:
sync.(*Mutex).Lock(0xc3c236c5e0)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3c1ef34a0, 0xae, 0xc2b75302d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3c1ef34a0, 0x1, 0xc3c24cc648, 0x64, 0xfc, 0xc3c240ef90, 0x40ec00, 0xc3c700e3c0, 0xc3cb46c3c0, 0xc3cb4686c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 420 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x5, 0xc23fc3d700, 0xea, 0x100, 0xc2401c2800, 0xbc, 0x100, 0xc24051f2c0, 0xc24051f200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 418 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12d, 0xc23fc3c300, 0xec, 0x100, 0xc23fc5a000, 0xcf, 0x100, 0xc23fcd74a0, 0xc23fcd73e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1686 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x0, 0xc23dc1c200, 0xea, 0x100, 0xc2cb51d800, 0xa4, 0x100, 0xc278b05da0, 0xc278b05ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 632 [chan send, 6 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2411be900, 0x1, 0xc24157cc00, 0xea, 0x100, 0xc23933f000, 0xc7, 0x100, 0xc23436f080, 0xc23436efc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 954 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0x12e, 0xc28a481200, 0xec, 0x100, 0xc28a1c7800, 0xb3, 0x100, 0xc28a9676e0, 0xc28a967620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5538 [runnable]:
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).checkReplicationActiveness(0xc426621980, 0xc20802c240, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:628
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc426621980, 0xc4268992c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:588 +0xd74
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc426621980)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 1776 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x67, 0xc3074bd500, 0xec, 0x100, 0xc307762000, 0xb3, 0x100, 0xc307755620, 0xc307755560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1013 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca7b00, 0xc9, 0xc27ca32400, 0xec, 0x100, 0xc287c42000, 0xbd, 0x100, 0xc284df2f60, 0xc284df2ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1004 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x65, 0xc27d2fb500, 0xec, 0x100, 0xc287f77000, 0xc6, 0x100, 0xc24c158c60, 0xc24c158ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1577 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0xc9, 0xc2a66c9200, 0xec, 0x100, 0xc2ba70c400, 0x5f, 0x80, 0xc2d962aa80, 0xc2d962a9c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4357 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea480, 0x64, 0xc3b9041100, 0xec, 0x100, 0xc398f1f800, 0xc9, 0x100, 0xc3db9a8a20, 0xc3db9a8960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1747 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x66, 0xc2f79c5800, 0xec, 0x100, 0xc300787000, 0xc8, 0x100, 0xc300d0d320, 0xc300d0d260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5794 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d71c0f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5695 [select]:
net/http.(*persistConn).roundTrip(0xc3e1a182c0, 0xc21c3f4dc0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc258b8e8f0, 0xc25b2ccc00, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc258b8e8f0, 0x7efee1818fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc258b8e8f0, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc258b8e8f0, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc258b8e8f0, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc258b8e8f0, 0x0, 0x818220, 0xc435e8fe58, 0xeb0d80, 0xc2080ec610, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc42c1e3d77, 0x11, 0x9ed550, 0xe, 0x0, 0xc20802bd40, 0xd, 0xc20802bd50, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080ec620, 0x9ed550, 0xe, 0xc2835e4d20, 0xeb0d80, 0x5, 0x66504c, 0x30000c435e8fe50, 0xc42c0b9ec0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080ec620, 0xc42c0c9260, 0xc24c38b480, 0xc2080b3e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc42c3824b0, 0x0, 0xc42c4e9080, 0x64, 0x160, 0xc42c1e3e90, 0xc3d4005f00, 0xc434d9e450, 0xc4351b77c0, 0xc4351b80c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 1033 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26bca78c0, 0x68, 0xc258b2ee00, 0xec, 0x100, 0xc299f25000, 0xc2, 0x100, 0xc29a223500, 0xc29a223440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1727 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0xc8, 0xc303fca600, 0xec, 0x100, 0xc3037e9000, 0xbb, 0x100, 0xc3040b2ba0, 0xc3040b2ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5561 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc41cb74d80, 0xc8, 0xc4274cb200, 0xec, 0x100, 0xc42757f800, 0x9f, 0x100, 0xc427924fc0, 0xc427924f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2741 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38d80, 0x64, 0xc23dc1d600, 0xec, 0x100, 0xc37f7a0000, 0xc9, 0x100, 0xc37f8d3140, 0xc37f8d3080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3520 [chan send]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).UpdateSettings(0xc3c1fbc900, 0xc3c2441ce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:628 +0x2b7
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).UpdateSettings(0xc3c223cb00, 0xc3c2441ce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:633 +0x2b0
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3c1ef34a0, 0x10e, 0xc2b7530180, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:392 +0x9b8
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3c1ef34a0, 0x2, 0xc3c24cc710, 0x64, 0x98, 0xc3c240ef90, 0x40ec00, 0xc3c700e3c0, 0xc3cb46c3c0, 0xc3cb4686c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 5645 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc42bf79c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1434 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d440, 0x6, 0xc28a481e00, 0xea, 0x100, 0xc2c4392000, 0xbe, 0x100, 0xc2c6a2e6c0, 0xc2c6a2e600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5627 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d715a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4083 [select]:
net/http.(*persistConn).writeLoop(0xc33a917600)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 5797 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d714c30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5684 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc42bfe18c0, 0xc42bfb5e30, 0xc42bfe1a60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 2752 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38fc0, 0xc8, 0xc221a78e00, 0xec, 0x100, 0xc389f2d800, 0xbb, 0x100, 0xc38a65bb60, 0xc38a65baa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3081 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38aeff440, 0xca, 0xc39c01fb00, 0xec, 0x100, 0xc39c4c5000, 0xbc, 0x100, 0xc39cbe58c0, 0xc39cbe5800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1345 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d680, 0x12f, 0xc2b9e60b00, 0xec, 0x100, 0xc2b95cd000, 0xce, 0x100, 0xc2ba4a3020, 0xc2ba4a2f60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4374 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea6c0, 0x12c, 0xc3acf95500, 0xec, 0x100, 0xc34859c800, 0xc4, 0x100, 0xc3ba923920, 0xc3ba923860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1419 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d680, 0xc8, 0xc221a79d00, 0xec, 0x100, 0xc2d5fc4000, 0xbb, 0x100, 0xc2dc7c5d40, 0xc2dc7c5c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3353 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9d40, 0xc8, 0xc3b9040d00, 0xec, 0x100, 0xc3b89d9800, 0xbb, 0x100, 0xc3b91c85a0, 0xc3b91c84e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1397 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d680, 0xca, 0xc2ca0e2200, 0xec, 0x100, 0xc2c95c8800, 0xbc, 0x100, 0xc2ca1d5ce0, 0xc2ca1d5c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1358 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d680, 0x12e, 0xc2ab7ae300, 0xec, 0x100, 0xc2ab792000, 0xb3, 0x100, 0xc2ab7dc6c0, 0xc2ab7dc600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5786 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·004(0x7efee1822b70, 0xc42bf79c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:362
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 1392 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d680, 0x130, 0xc2d5c40b00, 0xec, 0x100, 0xc2deec2800, 0xa7, 0x100, 0xc2e27924e0, 0xc2e2792420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1759 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4d80, 0xcc, 0xc305a75100, 0xec, 0x100, 0xc305c21000, 0xb7, 0x100, 0xc305f57f80, 0xc305f57ec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3085 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38aeff440, 0xc8, 0xc39cce5f00, 0xec, 0x100, 0xc39ed4a000, 0xbb, 0x100, 0xc39edc09c0, 0xc39edc0900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5798 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d68c370)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3334 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9b00, 0x64, 0xc3b0421200, 0xec, 0x100, 0xc3b13b2800, 0xc9, 0x100, 0xc3b18e1440, 0xc3b18e1380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5668 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc42bfe1680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2004 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48480, 0x12e, 0xc31b0db200, 0xec, 0x100, 0xc31abd0800, 0xb3, 0x100, 0xc31b371740, 0xc31b371680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5766 [select]:
net/http.(*persistConn).writeLoop(0xc3e1e1f8c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1680 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x1, 0xc20802ef00, 0xea, 0x100, 0xc26ac59800, 0xc7, 0x100, 0xc2ac9ab2c0, 0xc2ac9ab200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1735 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x65, 0xc2f2714300, 0xec, 0x100, 0xc2f47cb800, 0xc6, 0x100, 0xc2f4f42000, 0xc2f4e29f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1329 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c10d680, 0x130, 0xc2a66c9d00, 0xeb, 0x100, 0xc2a6c78900, 0x20, 0x20, 0xc2a6c25aa0, 0xc2a6c259e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2072 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48240, 0xa, 0xc356a13300, 0xeb, 0x100, 0xc355afd000, 0x1f4, 0x200, 0xc356de8120, 0xc356de8060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5694 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc42c0da000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 4002 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c20d7b00, 0xc8, 0xc208163e00, 0xec, 0x100, 0xc3725c7000, 0x9d, 0x100, 0xc2ef707320, 0xc2ef707260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5771 [IO wait]:
net.(*pollDesc).Wait(0xc32834fdb0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc32834fdb0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc32834fd50, 0xc24d219880, 0x18, 0x18, 0x0, 0x7efee1818df0, 0xc2169ab3c8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3e0c7d0d0, 0xc24d219880, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7efee181ab08, 0xc3e0c7d0d0, 0xc24d219880, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7efee181ab08, 0xc3e0c7d0d0, 0xc24d219880, 0x18, 0x18, 0xc4373a2200, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc269898c60, 0x7efee181ab08, 0xc3e0c7d0d0, 0xc24d219880, 0x18, 0x18, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:154 +0x11e
github.com/couchbase/gomemcached/client.getResponse(0x7efee181ab08, 0xc3e0c7d0d0, 0xc24d219880, 0x18, 0x18, 0xc269898c60, 0x60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Send(0xc25f54c7b0, 0xc269898c00, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:80 +0x10d
github.com/couchbase/gomemcached/client.(*Client).AuthList(0xc25f54c7b0, 0xc20802b8d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:123 +0x67
github.com/couchbase/gomemcached/client.(*Client).Auth(0xc25f54c7b0, 0xc20802b8e0, 0x6, 0xc20801fa20, 0x20, 0xc20801fa20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:128 +0x4b
github.com/couchbase/cbauth.func·005(0x7efee1819448, 0xc2080380e8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/mcdauthhandler.go:66 +0xf8
github.com/couchbase/cbauth.WithAuthenticator(0x7efee1819448, 0xc2080380e8, 0xc3222dfa70, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:123 +0x8a
github.com/couchbase/cbauth.(*AuthHandler).AuthenticateMemcachedConn(0xc24af3fc20, 0xc241072b80, 0x12, 0xc25f54c7b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/mcdauthhandler.go:71 +0x77
github.com/couchbase/go-couchbase.defaultMkConn(0xc241072b80, 0x12, 0x7efee181ba78, 0xc24af3fc20, 0x7efed8ccda00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:64 +0x162
github.com/couchbase/go-couchbase.(*connectionPool).GetWithTimeout(0xc35885dc80, 0x9356907420000, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:152 +0x635
github.com/couchbase/go-couchbase.(*connectionPool).Get(0xc35885dc80, 0xc2337f30f8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:165 +0x46
github.com/couchbase/go-couchbase.getStatsParallel(0xc23dcde340, 0x0, 0x9cac80, 0x3, 0xc2aa837d40)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:135 +0x10d
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 2056 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48480, 0xc8, 0xc348188e00, 0xec, 0x100, 0xc3476d9000, 0xbb, 0x100, 0xc348367260, 0xc3483671a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2607 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38fc0, 0x12e, 0xc2b2ecdc00, 0xec, 0x100, 0xc3115ef800, 0x65, 0x80, 0xc300b4e180, 0xc300b4e0c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2094 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48480, 0xca, 0xc357dbde00, 0xec, 0x100, 0xc358189000, 0xbc, 0x100, 0xc358595020, 0xc358594f60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3349 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9b00, 0x4, 0xc3b366f100, 0xea, 0x100, 0xc3b64d4000, 0xa2, 0x100, 0xc3b6546480, 0xc3b65463c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5680 [select]:
net/http.(*persistConn).writeLoop(0xc4352f2420)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3909 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c20d78c0, 0x67, 0xc3dbc81200, 0xec, 0x100, 0xc3dab22800, 0xb3, 0x100, 0xc3e0a0b320, 0xc3e0a0b260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5210 [select]:
net/http.(*persistConn).writeLoop(0xc40fa94210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2628 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38d80, 0x1, 0xc2ab7afa00, 0xe9, 0x100, 0xc2ec766600, 0x1e, 0x20, 0xc2efa0c720, 0xc2efa0c660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1668 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2e01b4b40, 0x64, 0xc24157dc00, 0xec, 0x100, 0xc2db028800, 0xc9, 0x100, 0xc2642fd980, 0xc2642fd8c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5795 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d70a910)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5686 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc42bfe18c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2082 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48480, 0xcb, 0xc35581e900, 0xec, 0x100, 0xc354fc1800, 0xc9, 0x100, 0xc3557cbc20, 0xc3557cbb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2710 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38d80, 0x0, 0xc2768c1100, 0xea, 0x100, 0xc36a390000, 0xb8, 0x100, 0xc36a47c1e0, 0xc36a47c120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2030 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48480, 0x12c, 0xc336cb4600, 0xec, 0x100, 0xc3365c9000, 0xc0, 0x100, 0xc336d8d800, 0xc336d8d740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4926 [semacquire]:
sync.(*Mutex).Lock(0xc3fa316af8)
	/usr/local/go/src/sync/mutex.go:66 +0xd3
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3f9d15950, 0x9f, 0xc24e318ab0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:387 +0x905
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3f9d15950, 0x1, 0xc3fe5fe388, 0x64, 0xfc, 0xc2c55cfa70, 0x0, 0xc2c51ab590, 0xc2b6f92e60, 0xc40320c9c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 2044 [chan send, 5 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc307d48240, 0x65, 0xc34c67cf00, 0xec, 0x100, 0xc34eb76000, 0x11a, 0x200, 0xc34ebe1140, 0xc34ebe1080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5800 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d715a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5768 [select]:
net/http.(*persistConn).writeLoop(0xc3e1a182c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4745 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc368ca6000, 0x0, 0xc27d2fa500, 0xea, 0x100, 0xc375681000, 0xb8, 0x100, 0xc3b26980c0, 0xc3b2698000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5796 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d71cff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5793 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d726460)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4423 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea6c0, 0xc8, 0xc3244fe700, 0xec, 0x100, 0xc3b2438000, 0xbb, 0x100, 0xc3777db3e0, 0xc3777db320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5792 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d68dcc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5212 [select]:
net/http.(*persistConn).writeLoop(0xc40f8624d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 5646 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc42bf79c80, 0xc430744b70)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 5765 [runnable]:
net.(*pollDesc).Wait(0xc31ff92140, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc31ff92140, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc31ff920e0, 0xc3da7d2000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc21e9d9f90)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc435e8e4e8, 0xc3da7d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc435e8e4e8, 0xc3e1e1f918, 0xc3da7d2000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2284ccc20, 0xc3da7d2000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2940369c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2940369c0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3e1e1f8c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 5625 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d68c370)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 5692 [runnable]:
io.ReadFull(0x7efee181ab08, 0xc42bf54990, 0xc2dc1a8d00, 0x826, 0x826, 0x826, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc23dab4e78, 0x7efee181ab08, 0xc42bf54990, 0xc434d8a500, 0x18, 0x18, 0xc289ccec7f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc434b44360, 0xc434b4ed80)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 5211 [IO wait]:
net.(*pollDesc).Wait(0xc40fa35a30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc40fa35a30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc40fa359d0, 0xc40ff06000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc225357fd0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc40db77320, 0xc40ff06000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc40db77320, 0xc40f862528, 0xc40ff06000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc25aea16c0, 0xc40ff06000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc40fe70840)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc40fe70840, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc40f8624d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4419 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea480, 0x1, 0xc333cbbc00, 0xea, 0x100, 0xc3bbca3000, 0xc7, 0x100, 0xc2778e67e0, 0xc2778e6720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4387 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea480, 0x65, 0xc3a00d9700, 0xec, 0x100, 0xc3384b1800, 0xc6, 0x100, 0xc39bd360c0, 0xc39bd36000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4352 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea6c0, 0xcb, 0xc3b9518a00, 0xec, 0x100, 0xc3b530b800, 0xab, 0x100, 0xc3ddd30e40, 0xc3ddd30d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3782 [select]:
net/http.(*persistConn).writeLoop(0xc3d634a370)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3361 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9b00, 0x1, 0xc3b9519c00, 0xea, 0x100, 0xc3bc02c000, 0xc7, 0x100, 0xc3bc2fbec0, 0xc3bc2fbe00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2788 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc347e38d80, 0x65, 0xc22dd69c00, 0xec, 0x100, 0xc38a740000, 0xc6, 0x100, 0xc38a779920, 0xc38a779860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3311 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9b00, 0x0, 0xc3acf95e00, 0xea, 0x100, 0xc3adc5c000, 0xb3, 0x100, 0xc3adc91140, 0xc3adc91080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5222 [IO wait]:
net.(*pollDesc).Wait(0xc40fa18220, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc40fa18220, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc40fa181c0, 0xc40fb83000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc40fec9408)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc40f9163b0, 0xc40fb83000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc40f9163b0, 0xc40f862688, 0xc40fb83000, 0x1000, 0x1000, 0x7efee18074c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc25b745120, 0xc40fb83000, 0x1000, 0x1000, 0xc1e786e9c3, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc40fe2f560)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc40fe2f560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc40f862630)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 5776 [runnable]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc42bfe1680, 0x64, 0xc20802e800, 0xec, 0x100, 0xc370210800, 0xc9, 0x100, 0xc269898b40, 0xc269898a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3781 [IO wait]:
net.(*pollDesc).Wait(0xc3d6789100, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3d6789100, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3d67890a0, 0xc3d6867000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc22c3daa60)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3d42f47c0, 0xc3d6867000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc3d42f47c0, 0xc3d634a3c8, 0xc3d6867000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3d6852fc0, 0xc3d6867000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3d68c9b00)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3d68c9b00, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3d634a370)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 5634 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d70a910)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3050 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38aeff440, 0x12c, 0xc39a4b4400, 0xec, 0x100, 0xc399e96800, 0xc4, 0x100, 0xc39a57ac00, 0xc39a57ab40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5789 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·004(0x7efee18227d0, 0xc42bfe18c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:362
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 5696 [select]:
net/http.(*persistConn).roundTrip(0xc4352f2420, 0xc21ef8a1e0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc259f0f6c0, 0xc25ba70060, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc259f0f6c0, 0x7efee1818fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc259f0f6c0, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc259f0f6c0, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc259f0f6c0, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc259f0f6c0, 0x0, 0x818220, 0xc435e8fdd0, 0xeb0d80, 0xc2080ec610, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc42c1e3d77, 0x11, 0x9ed550, 0xe, 0x0, 0xc20802bd40, 0xd, 0xc20802bd50, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080ec620, 0x9ed550, 0xe, 0xc2c0941800, 0xeb0d80, 0x5, 0x66504c, 0x30000c435e8fdc8, 0xc42c461880, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080ec620, 0xc42c0c9260, 0xc24c5f9720, 0xc2080b7e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc42c3824b0, 0x1, 0xc42c4e9148, 0x64, 0xfc, 0xc42c1e3e90, 0xc3d4005f00, 0xc434d9e450, 0xc4351b77c0, 0xc4351b80c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3384 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16f9d40, 0xc9, 0xc3c0324500, 0xec, 0x100, 0xc3bf2f2800, 0xbd, 0x100, 0xc3c037b440, 0xc3c037b380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5630 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d68d1d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 5773 [IO wait]:
net.(*pollDesc).Wait(0xc32834f4f0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc32834f4f0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc32834f490, 0xc24c4d6000, 0x18, 0x18, 0x0, 0x7efee1818df0, 0xc21e6b3918)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080387e0, 0xc24c4d6000, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7efee181ab08, 0xc2080387e0, 0xc24c4d6000, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7efee181ab08, 0xc2080387e0, 0xc24c4d6000, 0x18, 0x18, 0xc240ba8980, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc2a6cbc5a0, 0x7efee181ab08, 0xc2080387e0, 0xc24c4d6000, 0x18, 0x18, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:154 +0x11e
github.com/couchbase/gomemcached/client.getResponse(0x7efee181ab08, 0xc2080387e0, 0xc24c4d6000, 0x18, 0x18, 0xc2a6cbc5a0, 0x60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Send(0xc25b97c300, 0xc2a6cbc540, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:80 +0x10d
github.com/couchbase/gomemcached/client.(*Client).AuthList(0xc25b97c300, 0xc20801f860, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:123 +0x67
github.com/couchbase/gomemcached/client.(*Client).Auth(0xc25b97c300, 0xc20802b850, 0x6, 0xc20801f880, 0x20, 0xc20801f880, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:128 +0x4b
github.com/couchbase/cbauth.func·005(0x7efee1819448, 0xc2080380e8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/mcdauthhandler.go:66 +0xf8
github.com/couchbase/cbauth.WithAuthenticator(0x7efee1819448, 0xc2080380e8, 0xc3222e1a70, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:123 +0x8a
github.com/couchbase/cbauth.(*AuthHandler).AuthenticateMemcachedConn(0xc24af3fce0, 0xc241072c40, 0x1d, 0xc25b97c300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/mcdauthhandler.go:71 +0x77
github.com/couchbase/go-couchbase.defaultMkConn(0xc241072c40, 0x1d, 0x7efee181ba78, 0xc24af3fce0, 0x7efed8ccda00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:64 +0x162
github.com/couchbase/go-couchbase.(*connectionPool).GetWithTimeout(0xc35885dd00, 0x9356907420000, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:152 +0x635
github.com/couchbase/go-couchbase.(*connectionPool).Get(0xc35885dd00, 0xc2337f30f8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/conn_pool.go:165 +0x46
github.com/couchbase/go-couchbase.getStatsParallel(0xc23dcde340, 0x2, 0x9cac80, 0x3, 0xc2aa837d40)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:135 +0x10d
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 3975 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c20d7b00, 0xc8, 0xc3dbc80400, 0xec, 0x100, 0xc358e56000, 0x49, 0x80, 0xc267e4f8c0, 0xc267e4f800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4381 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea480, 0x2, 0xc34c67da00, 0xea, 0x100, 0xc3d0b26800, 0xcf, 0x100, 0xc31bba7020, 0xc31bba6f60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5647 [runnable]:
io.ReadFull(0x7efee181ab08, 0xc42be24e28, 0xc2d63cd600, 0x872, 0x872, 0x872, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc29be41e78, 0x7efee181ab08, 0xc42be24e28, 0xc4309fb3c0, 0x18, 0x18, 0xc41b60d91f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc42c47d680, 0xc43097eea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 5785 [runnable]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc42bfe18c0, 0xc8, 0xc436c59e00, 0xec, 0x100, 0xc37168b000, 0x63, 0x80, 0xc2a6cbc1e0, 0xc2a6cbc120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5801 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d70b590)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5689 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc434bd67e0, 0xc434b4ea20, 0xc434b4ea80)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 5644 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc430941e60, 0xc43097e060, 0xc43097e0c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 4407 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea6c0, 0x12d, 0xc33ec27a00, 0xec, 0x100, 0xc3cd688800, 0xcf, 0x100, 0xc36ba4c900, 0xc36ba4c840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5790 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d71dd10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5666 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc42bfe1680, 0xc42bfb5ce0, 0xc42bfe1820)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 4711 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc368ca6240, 0xc8, 0xc2ab7ae400, 0xec, 0x100, 0xc37810f000, 0xb2, 0x100, 0xc2a67a1ec0, 0xc2a67a1e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5799 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·005(0x7efee1822bf0, 0xc25d7272c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:373
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 5624 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d714c30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4282 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea6c0, 0xcb, 0xc3b366e300, 0xec, 0x100, 0xc3b35b5000, 0x7a, 0x80, 0xc3b5e569c0, 0xc3b5e56900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4772 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc368ca6000, 0x64, 0xc23dc1cf00, 0xec, 0x100, 0xc3abb5f800, 0xc9, 0x100, 0xc39d503800, 0xc39d503740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5693 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc42c0da000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:367 +0x2392
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 4431 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2dc7ea480, 0x6a, 0xc312390b00, 0xec, 0x100, 0xc380e15800, 0xcc, 0x100, 0xc37d41a540, 0xc37d41a480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5774 [runnable]:
net.(*pollDesc).Wait(0xc328f5e840, 0x77, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitWrite(0xc328f5e840, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:93 +0x43
net.(*netFD).connect(0xc328f5e7e0, 0x0, 0x0, 0x7efee1819a20, 0xc24c38bf60, 0xed0918ce2, 0x3354109c, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:114 +0x24b
net.(*netFD).dial(0xc328f5e7e0, 0x7efee18199d0, 0x0, 0x7efee18199d0, 0xc25b2b3170, 0xed0918ce2, 0x3354109c, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/net/sock_posix.go:139 +0x37a
net.socket(0x9d8aa0, 0x3, 0x2, 0x1, 0x0, 0xc25b2b3100, 0x7efee18199d0, 0x0, 0x7efee18199d0, 0xc25b2b3170, ...)
	/usr/local/go/src/net/sock_posix.go:91 +0x422
net.internetSocket(0x9d8aa0, 0x3, 0x7efee18199d0, 0x0, 0x7efee18199d0, 0xc25b2b3170, 0xed0918ce2, 0xc43354109c, 0xeb0f40, 0x1, ...)
	/usr/local/go/src/net/ipsock_posix.go:137 +0x148
net.dialTCP(0x9d8aa0, 0x3, 0x0, 0xc25b2b3170, 0xed0918ce2, 0x3354109c, 0xeb0f40, 0xc21c3f50e0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:156 +0x125
net.dialSingle(0x9d8aa0, 0x3, 0xc25b2b2247, 0x11, 0x0, 0x0, 0x7efee1819940, 0xc25b2b3170, 0xed0918ce2, 0x3354109c, ...)
	/usr/local/go/src/net/dial.go:235 +0x3f5
net.func·016(0xed0918ce2, 0xe3354109c, 0xeb0f40, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:163 +0x12c
net.dial(0x9d8aa0, 0x3, 0x7efee1819940, 0xc25b2b3170, 0xc2e4bb3b08, 0xed0918ce2, 0xe3354109c, 0xeb0f40, 0x0, 0x0, ...)
	/usr/local/go/src/net/fd_unix.go:40 +0x6f
net.(*Dialer).Dial(0xc20800a0c0, 0x9d8aa0, 0x3, 0xc25b2b2247, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:170 +0x4be
net.*Dialer.Dial·fm(0x9d8aa0, 0x3, 0xc25b2b2247, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:38 +0x79
net/http.(*Transport).dial(0xc20802c090, 0x9d8aa0, 0x3, 0xc25b2b2247, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:479 +0x84
net/http.(*Transport).dialConn(0xc20802c090, 0x0, 0xc25b2b2240, 0x4, 0xc25b2b2247, 0x11, 0x4, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:564 +0x1678
net/http.func·019()
	/usr/local/go/src/net/http/transport.go:520 +0x42
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:522 +0x335

goroutine 5690 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc42c0da000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 5648 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc42bf79c80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:367 +0x2392
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 4709 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc368ca6000, 0x2, 0xc2b2ecde00, 0xea, 0x100, 0xc37810d800, 0xcf, 0x100, 0xc2f3febd40, 0xc2f3febc80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4735 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc368ca6240, 0xca, 0xc288644700, 0xec, 0x100, 0xc36c845000, 0xbc, 0x100, 0xc3af7cd740, 0xc3af7cd680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5649 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc42bf79c80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 5685 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc42bfe18c0, 0xa900c8, 0xc3bd8f1b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:836 +0x14d1
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc42bfe18c0, 0xc292cc00c8, 0xc3bd8f1b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc42bfe18c0, 0xc434b2efc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc42bfe18c0, 0xc434b2efc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc42bfe18c0, 0xc42bfb5d50, 0xc42bfe1a60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 4801 [semacquire]:
sync.(*WaitGroup).Wait(0xc2b6f92e60)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc3f9d15950, 0xc2080ea380, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc20814e140, 0x7efee18228b0, 0xc3f9ef4790, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7efee18228b0, 0xc3f9ef4790, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 5760 [chan receive]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc23dcde340, 0x9cac80, 0x3, 0xc2080ec408)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc42c0b9040, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc42c0b9040, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3ac3c50e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 5784 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).tcpProxy(0xc42bfe18c0, 0xc8, 0xc2a6cbc1e0, 0xc32834f8f0, 0xc2a6cbc120, 0xc2a6cbc180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:902 +0x17da
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:831 +0xc91

goroutine 5209 [IO wait]:
net.(*pollDesc).Wait(0xc40fa35950, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc40fa35950, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc40fa358f0, 0xc40fe7a000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc40fe786a0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc40db77308, 0xc40fe7a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc40db77308, 0xc40fa94268, 0xc40fe7a000, 0x1000, 0x1000, 0x7efee1807010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc25aea12c0, 0xc40fe7a000, 0x1000, 0x1000, 0xc1e7a488cd, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc40fe70780)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc40fe70780, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc40fa94210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 5667 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc42bfe1680, 0xa90064, 0xc3bbf8db50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:836 +0x14d1
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc42bfe1680, 0xc25ae90064, 0xc3bbf8db50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc42bfe1680, 0xc430737260, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc42bfe1680, 0xc430737260, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc42bfe1680, 0xc42bfb5c00, 0xc42bfe1820, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 5027 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3f9f31680, 0xc8, 0xc4055c4300, 0xec, 0x100, 0xc4051db000, 0xaa, 0x100, 0xc405bd8ae0, 0xc405b8be60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5622 [semacquire]:
sync.(*WaitGroup).Wait(0xc4351b77c0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc42c3824b0, 0xc2080ea380, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc20814e140, 0x7efee18228b0, 0xc42bfcf1e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7efee18228b0, 0xc42bfcf1e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 5679 [runnable]:
net.(*pollDesc).Wait(0xc4352bed10, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc4352bed10, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc4352becb0, 0xc4353f8000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc21e9d9f38)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc42bf54e20, 0xc4353f8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc42bf54e20, 0xc4352f2478, 0xc4353f8000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc435241700, 0xc4353f8000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc4352ed3e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc4352ed3e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc4352f2420)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 5788 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·004(0x7efee1822b70, 0xc42c0da000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:362
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 5626 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d7272c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 5629 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d71dd10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 5767 [runnable]:
net.(*pollDesc).Wait(0xc3204767d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3204767d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc320476770, 0xc3dd0bb000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc23d239b58)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc435e8e4f0, 0xc3dd0bb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7efee181ab08, 0xc435e8e4f0, 0xc3e1a18318, 0xc3dd0bb000, 0x1000, 0x1000, 0x7efee1807010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2284ccd60, 0xc3dd0bb000, 0x1000, 0x1000, 0xc1e52171af, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc294036a20)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc294036a20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3e1a182c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 5623 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25d71cff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 5496 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc41cb74b40, 0x0, 0xc426016f00, 0xe9, 0x100, 0xc41c711740, 0x8, 0x8, 0xc4260663c0, 0xc426066300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 5698 [select]:
net/http.(*Transport).getConn(0xc20802c090, 0xc258b8edd0, 0x0, 0xc25b2b2240, 0x4, 0xc25b2b2247, 0x11, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:525 +0x608
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc258b8edd0, 0xc25b2b2de0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:228 +0x4d4
net/http.send(0xc258b8edd0, 0x7efee1818fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc258b8edd0, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc258b8edd0, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc258b8edd0, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc258b8edd0, 0x0, 0x818220, 0xc435e8fe98, 0xeb0d80, 0xc2080ec610, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc42c1e3d77, 0x11, 0x9ed550, 0xe, 0x0, 0xc20802bd40, 0xd, 0xc20802bd50, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080ec620, 0x9ed550, 0xe, 0xc2835e4f00, 0xeb0d80, 0x5, 0x66504c, 0x30000c435e8fe90, 0xc42c4709c0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080ec620, 0xc42c0c9260, 0xc24c38b9e0, 0xc2e4bb1e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc42c3824b0, 0x3, 0xc42c4e92d8, 0x29, 0x34, 0xc42c1e3e90, 0xc3d4005f00, 0xc434d9e450, 0xc4351b77c0, 0xc4351b80c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 5787 [runnable]:
github.com/couchbase/goxdcr/pipeline.func·004(0x7efee18227d0, 0xc42bfe1680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:362
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 5573 [IO wait]:
net.(*pollDesc).Wait(0xc428cb9330, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc428cb9330, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc428cb92d0, 0xc313b84000, 0x1000, 0x1000, 0x0, 0x7efee1818df0, 0xc217cf0538)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc427775710, 0xc313b84000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc4282d47c8, 0xc313b84000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc42903bbc0, 0xc313b84000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc320f67f80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc320f67f80, 0xc240ba870a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc320f67f80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc2394e95f0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc2394e95f0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc320f67f80, 0xc3e1d86750, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc4282d4780, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc4282d4780)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e
[goport] 2017/04/25 13:30:30 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:30:30.738-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:30:30.738-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:30:30.738-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:30:30.760-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:30:30.762-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:30:30.763-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:30:30.763-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:30:30.763-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:30:30.763-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:30:30.763-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:30:30.763-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:30:30.763-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:30:30.763-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:30:30.763-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:30:30.763-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:30:30.763-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:30:30.763-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:30:30.763-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:30:30.763-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:30:30.763-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:30:30.763-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:30:30.763-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:30:30.763-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:30:30.764-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:30:30.766-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:30:30.767-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:30:30.767-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:30:30.767-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:30:31.396-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:30:45.809-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:30:45.814-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:30:45.814-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:30:45.814-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:30:45.815-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:30:45.815-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:30:45.815-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:30:45.815-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:30:45.815-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:30:45.916-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:30:45.916-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:30:45.916-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:30:45.969-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:30:45.970-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:30:45.970-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:30:45.970-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:30:45.970-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:30:45.970-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:30:45.970-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:30:45.974-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:30:45.977-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
GenericSupervisor 2017-04-25T13:30:45.979-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:30:45.979-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
GenericSupervisor 2017-04-25T13:30:45.979-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:45.979-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:45.979-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:45.979-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:30:45.979-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:30:45.979-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
CheckpointManager 2017-04-25T13:30:45.979-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
CapiUtils 2017-04-25T13:30:46.014-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:30:46.014-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:30:46.014-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:30:46.015-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
StatisticsManager 2017-04-25T13:30:46.016-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:30:46.016-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
XDCRFactory 2017-04-25T13:30:46.016-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:30:46.016-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125 is constructed, start it
GenericPipeline 2017-04-25T13:30:46.017-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[48 114 112 142 147 57 74 30 134 87 93 99 58 61 103 165 2 90 128 139 32 97 100 168 40 62 158 13 84 63 107 109 118 146 17 22 55 102 123 35 50 108 150 6 166 170 148 60 98 120 130 77 81 156 110 7 31 79 126 29 59 101 116 122 92 132 149 9 44 137 45 117 18 20 39 89 16 143 162 27 115 164 0 4 95 153 42 133 24 33 43 82 111 21 51 155 167 34 127 136 138 56 66 10 26 71 80 83 159 49 64 67 73 125 152 161 46 70 140 154 38 69 131 141 169 3 12 37 53 105 121 11 119 19 75 47 113 78 96 106 23 135 1 54 68 52 151 8 85 88 91 104 14 76 5 72 160 28 41 94 145 163 25 36 124 65 86 157 15 129 144] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[281 173 319 326 172 198 209 195 203 293 333 226 316 250 268 298 224 299 314 296 235 264 337 309 176 233 329 216 278 303 204 220 280 283 311 315 330 208 322 292 307 312 320 185 294 197 262 171 218 181 196 178 253 194 239 313 266 273 301 304 231 308 199 236 251 257 210 276 302 180 295 325 177 190 265 305 318 189 270 291 338 191 221 232 324 207 215 275 285 274 193 211 256 261 238 212 267 217 300 188 175 229 284 282 288 213 246 201 260 289 332 240 263 328 186 237 287 323 335 183 187 255 258 244 334 219 279 290 245 259 272 223 339 228 331 336 269 306 174 184 202 230 286 249 317 321 200 252 277 254 192 205 214 241 340 271 327 182 206 243 297 225 310 247 248 179 227 242 222 234]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[25 36 124 65 86 157 15 129 144 48 114 112 142 147 57 74 30 134 87 93 99 58 61 103 165 2 90 128 139 32 97 100 168 40 62 158 13 84 63 107 109 118 146 17 22 55 102 123 35 50 108 150 6 166 170 148 60 98 120 130 77 81 156 110 7 31 79 29 59 101 116 122 126 92 132 149 9 44 137 45 117 18 20 39 89 16 143 162 27 115 164 0 4 95 153 42 24 33 43 82 111 133 21 51 155 167 34 127 136 138 56 66 10 26 71 80 83 49 64 67 73 125 159 152 161 46 70 140 154 38 69 131 3 12 37 53 105 141 169 121 11 119 19 75 47 113 78 96 106 23 135 1 54 68 52 151 8 85 88 91 104 14 76 5 72 160 28 41 94 145 163] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[225 310 179 227 247 248 242 222 234 281 173 319 326 172 198 209 195 203 293 226 316 333 298 224 250 268 299 314 296 235 264 337 176 233 309 216 278 303 329 280 204 220 283 311 315 330 208 322 292 307 312 320 185 294 197 262 171 218 178 181 196 194 239 253 266 273 301 304 313 308 231 199 236 251 257 210 276 180 295 302 177 190 265 325 189 270 291 305 318 191 221 232 324 338 207 215 275 285 193 211 256 261 274 238 212 267 188 217 300 175 229 284 282 213 288 201 246 260 289 332 328 240 263 237 186 335 183 187 255 287 323 258 244 334 219 279 290 245 259 272 223 228 331 336 339 174 184 202 269 306 230 286 249 317 321 200 252 277 192 205 254 214 241 340 182 206 243 271 327 297]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:30:46.017-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125...
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:30:46.026-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:30:46.026-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:30:46.026-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:30:46.026-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:30:46.026-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:30:46.026-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:30:46.026-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:30:46.026-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:30:46.026-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:30:46.027-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:30:46.037-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:30:46.037-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:30:46.037-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:30:46.037-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:30:46.037-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:30:46.086-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:30:46.086-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:30:46.086-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:30:46.186-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:30:46.202-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:30:46.202-05:00 [INFO] connectionSize=5
ToplogyChangeDetector 2017-04-25T13:30:46.205-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:30:46.210-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
TCPConnPoolMgr 2017-04-25T13:30:46.212-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:30:46.212-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:30:46.212-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:30:46.212-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:30:46.221-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:30:46.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:30:46.269-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:30:46.280-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:30:46.280-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:30:46.280-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:30:46.280-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:30:46.280-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:30:46.281-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:30:46.369-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:30:46.369-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:30:46.370-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:30:46.383-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:30:46.383-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:30:46.539-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:30:46.539-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:30:46.539-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:30:46.539-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:30:46.539-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:30:46.539-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:30:46.540-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125 is started----------
PipelineManager 2017-04-25T13:30:46.540-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:30:46.540-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:30:46.540-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:30:47.113-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:30:48.146-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:30:49.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:30:50.131-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:30:51.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:30:52.254-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:30:52.435-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:30:52.435-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:30:52.435-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:30:52.436-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:30:52 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 109 successful
CapiNozzle 2017-04-25T13:30:52.792-05:00 [INFO] Send batch count=209
2017/04/25 13:30:52 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:30:52.918-05:00 [INFO] Send batch count=3
2017/04/25 13:30:52 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:30:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 151 successful
CapiNozzle 2017-04-25T13:30:53.107-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:30:53 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 57 successful
CapiNozzle 2017-04-25T13:30:53.128-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:30:53.141-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:30:53.167-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:30:53.167-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:30:53 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 273 successful
CapiNozzle 2017-04-25T13:30:53.187-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
StatisticsManager 2017-04-25T13:30:53.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:30:53.193-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:30:53 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 158 successful
CapiNozzle 2017-04-25T13:30:53.218-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:30:53.243-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:30:53.244-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:30:53.244-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:30:53.244-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
ReplicationManager 2017-04-25T13:30:53.244-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
PipelineManager 2017-04-25T13:30:53.263-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:30:53.264-05:00 [INFO] Send batch count=500
2017/04/25 13:30:53 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 286 successful
PipelineManager 2017-04-25T13:30:53.269-05:00 [INFO] err_list=[{"time":"2017-04-25T13:30:53.269299326-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:30:53.269-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:30:53 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 183 successful
CapiNozzle 2017-04-25T13:30:53.367-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:53.367-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:53.379-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:30:53 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 292 successful
CheckpointManager 2017-04-25T13:30:53.478-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
CheckpointManager 2017-04-25T13:30:53.478-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:30:53.481-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:30:53 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:30:53 UPR_STREAMREQ for vb 196 successful
CapiNozzle 2017-04-25T13:30:53.598-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:30:53.614-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
DcpNozzle 2017-04-25T13:30:53.650-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:30:53.851-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:30:53.879-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:30:54.039-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:30:54.040-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125 {"Errors": "[{\"time\":\"2017-04-25T13:30:53.269299326-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 83, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 4903, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 83, "set_filtered": 0, "set_received_from_dcp": 4903, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:30:54.119-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:30:54.141-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:30:54.251-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:30:54.285-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:30:54.456-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:30:54.456-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:30:54.456-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:30:54.456-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:54.456-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:30:54.548-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:54.548-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:54.559-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:30:54.605-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:30:54.605-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:30:54.605-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:30:54.605-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:54.605-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:30:54.692-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:54.692-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:54.703-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:30:54.939-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:30:54.951-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:30:55.312-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:30:55.343-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:30:55.423-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:30:55.446-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:30:55.713-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:30:55.729-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:30:55.979-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:30:55.979-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:30:55.979-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:30:55.980-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:55.980-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:30:56.063-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:30:56.063-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:30:56.065-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:30:56.074-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:56.074-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:56.085-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:30:56.223-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:30:56.223-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:30:56.223-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:30:56.223-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:56.223-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:30:56.375-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
ToplogyChangeDetector 2017-04-25T13:30:56.376-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:30:56.384-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:56.384-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:56.406-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:30:56.428-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:30:56.706-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:30:56.726-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:30:57.055-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:30:57.093-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:30:57.173-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:30:57.173-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:30:57.173-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:30:57.173-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:57.173-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:30:57.295-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:57.295-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:57.327-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:30:57.709-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:30:57.723-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:30:58.210-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:30:58.245-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:30:58.516-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:30:58.516-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:30:58.516-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:30:58.516-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:58.516-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:30:58.635-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:58.635-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:58.648-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:30:58.809-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:30:58.828-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:30:59.336-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:30:59.363-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:30:59.363-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:30:59.363-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:30:59.363-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:30:59.364-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:30:59.368-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:30:59.517-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:30:59.517-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:30:59.554-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:30:59.803-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:30:59.819-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:31:00.377-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:00.391-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
PipelineManager 2017-04-25T13:31:00.860-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:30:53.269299326-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]}
]
CapiNozzle 2017-04-25T13:31:00.894-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:00.894-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:31:00.894-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:31:00.894-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:00.894-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:01.031-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:01.031-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:01.053-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:31:01.302-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:01.342-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:31:01.862-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:01.937-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:31:02.332-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:02.332-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:31:02.332-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:31:02.332-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:02.332-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:02.438-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:02.438-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:02.455-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:31:02.609-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:02.622-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
PipelineManager 2017-04-25T13:31:03.385-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:31:03.385-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:31:03.385-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:03.385-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
PipelineRuntimeCtx 2017-04-25T13:31:03.385-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:31:03.385-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:31:03.385-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
GenericSupervisor 2017-04-25T13:31:03.385-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:31:03.385-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:03.385603524 -0500 CDT
GenericSupervisor 2017-04-25T13:31:03.388-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:03.388-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:31:03.388-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:03.388-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:31:03.388-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:03.388-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:31:03.388-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:31:03.388-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:31:03.388-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:31:03.388-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-970919125 has been stopped
GenericPipeline 2017-04-25T13:31:03.388-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-970919125 is stopped
PipelineManager 2017-04-25T13:31:03.388-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:31:03.388-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:30:53.269299326-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:31:03.388-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:03.388-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
GenericPipeline 2017-04-25T13:31:03.388-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
CapiNozzle 2017-04-25T13:31:03.389-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:03.389-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:31:03.389-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:31:03.389-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=4
CapiNozzle 2017-04-25T13:31:03.389-05:00 [INFO] Send batch count=500
GenericPipeline 2017-04-25T13:31:03.404-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
GenericPipeline 2017-04-25T13:31:03.404-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-970919125
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:03.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:03.405-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:31:03.405-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:03.405-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:31:03.405-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:03.405-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:31:03.405-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:03.405-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:31:03.405-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:03.405-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:03.405-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:31:03.406-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:03.405491843 -0500 CDT
DcpNozzle 2017-04-25T13:31:03.406-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:31:03.406-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:03.405799743 -0500 CDT
CapiNozzle 2017-04-25T13:31:03.406-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:03.406-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.406-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:03.407-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:03.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:31:03.409-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:03.409-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:03.409-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:03.409503116 -0500 CDT
DcpNozzle 2017-04-25T13:31:03.409-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:03.409513846 -0500 CDT
DcpNozzle 2017-04-25T13:31:03.409-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:31:03.409-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:31:03.409-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:03.409-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:31:03.409-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:03.409-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:31:03.411-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:31:03.480-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:31:03.480-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:31:03.480-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:03.480-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:31:03.494-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:31:03.494-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:31:03.494-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:03.494-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:31:03.555-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:03.555-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:31:03.555-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:31:03.693-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:03.693-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:31:03.693-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:03.693-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:31:03.694-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:03.694-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:31:03.694-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:31:03.694-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:31:03.695-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
GenericSupervisor 2017-04-25T13:31:03.696-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:31:03.697-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
GenericSupervisor 2017-04-25T13:31:03.697-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:03.697-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:03.697-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:03.697-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:03.697-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:31:03.697-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
CheckpointManager 2017-04-25T13:31:03.697-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
CapiUtils 2017-04-25T13:31:03.789-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:03.789-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:03.789-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:31:03.790-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
StatisticsManager 2017-04-25T13:31:03.790-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:31:03.790-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
XDCRFactory 2017-04-25T13:31:03.791-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:31:03.791-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885 is constructed, start it
GenericPipeline 2017-04-25T13:31:03.792-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[100 24 59 105 109 103 35 90 118 139 151 23 42 52 61 15 72 85 10 38 68 123 166 5 14 25 108 131 133 33 47 37 62 63 116 80 83 149 127 27 82 137 31 64 99 124 55 67 113 126 20 65 69 75 88 119 160 156 30 129 152 58 95 145 39 1 134 73 4 34 115 142 148 11 71 87 147 165 3 19 96 136 141 17 146 12 49 50 98 18 26 97 143 40 70 121 44 56 157 158 170 22 93 125 2 36 48 128 21 45 84 91 106 107 114 28 46 51 54 122 161 81 150 120 144 117 167 7 78 89 104 6 8 29 140 74 135 169 86 153 102 112 168 77 94 138 159 0 13 132 163 16 110 130 155 9 92 164 43 57 66 79 162 101 111 32 41 53 60 76 154] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[190 239 269 280 267 282 335 336 314 174 306 309 238 318 284 233 242 175 253 295 311 313 171 187 322 275 193 260 266 296 317 200 219 229 286 291 216 228 240 298 303 326 204 287 301 173 182 176 329 215 237 288 333 181 192 312 195 205 334 203 268 319 330 198 310 214 294 307 325 209 285 305 227 264 316 340 191 178 186 234 235 273 292 202 232 332 231 321 338 179 180 261 221 224 230 241 274 324 262 201 210 217 244 331 185 263 199 308 290 225 255 278 299 277 177 206 212 245 272 243 323 328 249 293 304 337 270 189 289 320 223 281 188 220 254 172 184 218 236 194 283 183 196 265 300 207 250 256 315 208 226 279 213 247 251 257 297 327 252 258 222 246 271 197 339 248 211 259 276 302]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[7 78 89 104 6 8 29 140 74 135 169 86 153 102 112 168 77 94 138 159 0 13 132 163 16 110 130 155 9 92 164 162 43 57 66 79 101 111 76 154 32 41 53 60 100 24 59 105 109 103 151 35 90 118 139 23 42 52 61 15 72 85 166 10 38 68 123 131 133 5 14 25 108 33 47 37 62 63 116 80 83 149 127 27 82 137 31 64 99 124 55 67 113 126 88 119 160 20 65 69 75 156 30 129 152 58 95 145 39 1 134 73 148 4 34 115 142 165 11 71 87 147 141 3 19 96 136 17 146 12 49 50 98 18 26 97 143 40 70 121 170 44 56 157 158 22 93 125 2 36 48 128 21 45 84 91 106 107 114 122 161 28 46 51 54 81 150 120 144 117 167] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[188 220 254 184 218 236 172 194 283 183 300 196 265 256 315 207 250 208 226 279 213 247 327 251 257 297 252 222 246 258 271 197 339 211 248 259 276 302 190 239 269 280 336 267 282 335 314 174 306 309 318 238 233 242 284 311 313 175 253 295 171 187 322 193 260 266 275 296 317 229 286 291 200 219 298 303 326 216 228 240 287 301 204 182 173 176 329 237 288 333 215 181 192 312 195 205 334 268 319 330 203 198 310 214 294 307 325 209 285 305 227 264 316 340 178 186 191 234 235 273 292 332 202 232 231 321 338 179 180 261 230 241 274 221 224 324 262 331 201 210 217 244 185 263 308 199 225 255 278 290 299 206 212 245 277 177 243 272 323 328 304 337 249 293 270 189 289 320 223 281]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info stats_interval:1000 active:true checkpoint_interval:1800 worker_batch_size:500 target_nozzle_per_node:2 replication_type:capi filter_expression:]
CheckpointManager 2017-04-25T13:31:03.792-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885...
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:03.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:31:03.802-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:31:03.802-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:31:03.802-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:03.802-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:31:03.802-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:31:03.802-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:31:03.802-05:00 [INFO] Checkpointing starts in 1802.988 sec
PipelineRuntimeCtx 2017-04-25T13:31:03.802-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:31:03.802-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:31:03.803-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:31:03.803-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:31:03.806-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:31:03.806-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:31:03.808-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:31:03.840-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:31:03.840-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:31:03.840-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:31:03.856-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:31:03.856-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:31:03.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:31:03.880-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:03.880-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:03.880-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:31:03.885-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:31:03.889-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:31:03.893-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:31:03.893-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:03.893-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:31:03.893-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:31:03.893-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:31:03.894-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
AsyncListener 2017-04-25T13:31:03.919-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:31:03.919-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:31:03.919-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiNozzle 2017-04-25T13:31:03.920-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:31:03.920-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:31:03.920-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:31:03.949-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:31:03.949-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:31:04.084-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:31:04.145-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:31:04.145-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:04.145-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:31:04.146-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:31:04.146-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:31:04.146-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:31:04.146-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885 is started----------
PipelineManager 2017-04-25T13:31:04.146-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:31:04.146-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:31:04.146-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:31:04.414-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:31:04.414-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:31:04.414-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:31:04.414-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:31:04 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:31:04.519-05:00 [INFO] Send batch count=56
2017/04/25 13:31:04 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:31:04.572-05:00 [INFO] Send batch count=121
2017/04/25 13:31:04 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:31:04.612-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:04.612-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:04 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:31:04.629-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:31:04 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 313 successful
CapiNozzle 2017-04-25T13:31:04.785-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:04.798-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:31:04 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 15 successful
StatisticsManager 2017-04-25T13:31:04.877-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:31:04 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:31:04 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 31 successful
CapiNozzle 2017-04-25T13:31:05.009-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:31:05 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 30 successful
CapiNozzle 2017-04-25T13:31:05.034-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:31:05 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 254 successful
CapiNozzle 2017-04-25T13:31:05.304-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:31:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 155 successful
CapiNozzle 2017-04-25T13:31:05.341-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:31:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 280 successful
CapiNozzle 2017-04-25T13:31:05.578-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:05.595-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:31:05 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 81 successful
CapiNozzle 2017-04-25T13:31:05.624-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:05.638-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:31:05 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:31:05.717-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
CheckpointManager 2017-04-25T13:31:05.717-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:31:05 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:31:05 UPR_STREAMREQ for vb 296 successful
DcpNozzle 2017-04-25T13:31:05.807-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:31:05.858-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:31:05.879-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:31:06.252-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:06.252-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:31:06.252-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:31:06.252-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:31:06.252-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:31:06.273-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:31:06.274-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:31:06.276-05:00 [INFO] err_list=[{"time":"2017-04-25T13:31:06.276643058-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:31:06.276-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:31:06.299-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:06.299-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:31:06.299-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:31:06.299-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:06.299-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:06.345-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:06.345-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:06.356-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:31:06.435-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:06.435-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:06.460-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
StatisticsManager 2017-04-25T13:31:06.810-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:31:06.810-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885 {"Errors": "[{\"time\":\"2017-04-25T13:31:06.276643058-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 76, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 55640, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 76, "set_filtered": 0, "set_received_from_dcp": 55640, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:31:06.928-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:31:06.954-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:31:07.211-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:07.250-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:31:08.133-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:31:08.134-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:08.191-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:31:08.215-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:31:08.830-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:08.830-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:31:08.830-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:31:08.830-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:08.830-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:08.964-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:08.964-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:08.996-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:31:09.192-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:31:09.192-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:31:09.192-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:31:09.192-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:09.192-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:09.263-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:09.263-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:09.276-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:31:09.357-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:09.369-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:31:09.663-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:09.674-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:31:10.072-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:10.091-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:31:10.229-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:10.256-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:31:10.462-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:10.462-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:31:10.462-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:31:10.462-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:10.462-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:10.545-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:10.545-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:10.562-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:31:10.702-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:10.702-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:31:10.702-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:31:10.702-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:10.702-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:10.854-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:10.854-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:10.873-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:31:10.922-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:10.947-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:31:11.292-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:11.304-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:31:11.467-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:11.492-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:31:11.643-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:11.675-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:31:11.832-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:11.832-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:31:11.832-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:31:11.832-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:11.832-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:12.001-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:12.001-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:12.019-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:31:12.342-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:12.342-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:31:12.342-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:31:12.342-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:12.342-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:12.439-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:12.450-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:12.463-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:31:12.552-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:12.574-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:31:12.840-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:12.850-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:31:13.101-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:13.112-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:31:13.400-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:13.411-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:31:13.569-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:13.569-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:31:13.569-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:31:13.569-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:13.569-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:13.649-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:13.649-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:13.660-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:31:13.801-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:13.801-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:31:13.801-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:31:13.801-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:13.801-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:31:13.806-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:31:13.806-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:31:13.806-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:31:13.885-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:13.885-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:13.896-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:31:14.103-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:14.114-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
ToplogyChangeDetector 2017-04-25T13:31:14.135-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:31:14.256-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:31:14.267-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:31:14.497-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:14.507-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:31:14.696-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:31:14.707-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:31:14.933-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:14.933-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:31:14.933-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:31:14.933-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:14.933-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:15.033-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:15.033-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:15.048-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:31:15.127-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:31:15.127-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:31:15.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:31:15.127-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:15.127-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:15.208-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:15.208-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:15.219-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:31:15.454-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:15.465-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:31:15.755-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:15.766-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:31:15.863-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:06.276643058-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:31:16.069-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:16.080-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
PipelineManager 2017-04-25T13:31:16.276-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:31:16.277-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:31:16.277-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:16.277-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
PipelineRuntimeCtx 2017-04-25T13:31:16.277-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:31:16.277-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:16.277-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:31:16.277-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:16.277203596 -0500 CDT
GenericSupervisor 2017-04-25T13:31:16.277-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:16.277-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:31:16.277-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:16.277-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:31:16.277-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:16.277-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:31:16.277-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:31:16.277-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:31:16.277-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:31:16.277-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-694144885 has been stopped
GenericPipeline 2017-04-25T13:31:16.277-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694144885 is stopped
PipelineManager 2017-04-25T13:31:16.277-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:31:16.278-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:06.276643058-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:31:16.278-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:16.278-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
GenericPipeline 2017-04-25T13:31:16.278-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
GenericPipeline 2017-04-25T13:31:16.278-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
GenericPipeline 2017-04-25T13:31:16.278-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-694144885
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:16.278-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:16.279-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:31:16.279-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:16.279-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:31:16.279-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:16.279-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:31:16.279-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:16.279-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:31:16.279-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:16.279-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:16.280-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:31:16.280-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:16.279573814 -0500 CDT
DcpNozzle 2017-04-25T13:31:16.280-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:31:16.280-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:16.279876252 -0500 CDT
CapiNozzle 2017-04-25T13:31:16.280-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:31:16.280-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:16.281-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:31:16.281-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:16.281-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:31:16.281-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:16.282-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:16.282-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:31:16.282-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:16.282-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:31:16 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:31:16.287-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:16.287-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:16.287532104 -0500 CDT
DcpNozzle 2017-04-25T13:31:16.287-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:31:16.287-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:16.287-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:31:16 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:31:16.299-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:16.299-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:16.299905202 -0500 CDT
DcpNozzle 2017-04-25T13:31:16.299-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:31:16.300-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:16.300-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:31:16.405-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:16.405-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:31:16.406-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:31:16.488-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:16.489-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:31:16.489-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:16.489-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:31:16.489-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:16.490-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:31:16.490-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:31:16.490-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:31:16.490-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
GenericSupervisor 2017-04-25T13:31:16.493-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:31:16.493-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
GenericSupervisor 2017-04-25T13:31:16.493-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:16.493-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:16.493-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:16.493-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:16.493-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:31:16.493-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
CheckpointManager 2017-04-25T13:31:16.493-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
CapiUtils 2017-04-25T13:31:16.583-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:16.583-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:16.583-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:31:16.584-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
StatisticsManager 2017-04-25T13:31:16.585-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:31:16.585-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
XDCRFactory 2017-04-25T13:31:16.585-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:31:16.585-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047 is constructed, start it
GenericPipeline 2017-04-25T13:31:16.586-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[114 169 23 28 42 94 8 90 36 147 89 100 103 111 125 149 165 53 137 151 17 47 159 21 120 69 58 84 107 110 18 98 37 164 157 46 106 14 7 88 158 43 25 56 65 67 148 153 39 79 122 27 44 71 93 124 168 0 33 105 138 155 24 115 131 146 29 83 166 22 87 92 45 95 132 134 59 145 78 85 99 19 121 167 4 142 10 116 74 41 54 64 12 113 150 170 81 34 141 1 140 161 2 50 86 136 20 144 38 82 160 40 126 130 112 77 32 15 133 156 48 80 108 70 6 135 109 61 75 139 143 3 129 52 31 118 152 163 60 102 119 162 35 101 117 123 16 30 51 57 91 13 128 73 76 96 72 127 62 11 26 154 5 97 9 55 66 49 68 104 63] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[184 188 215 308 309 193 207 225 272 293 335 247 251 334 212 238 211 264 316 331 217 220 230 181 187 250 171 268 284 306 321 332 244 248 269 311 245 222 239 262 263 275 283 333 323 231 286 305 202 246 253 312 226 243 257 219 249 318 288 337 213 177 216 197 295 317 324 223 235 314 327 330 259 303 172 260 278 301 191 255 267 276 292 194 203 338 180 210 304 196 271 319 190 273 340 174 182 175 236 280 289 287 185 229 328 277 195 279 192 237 240 326 206 300 315 176 282 329 336 200 290 296 204 227 252 281 233 234 218 265 297 339 183 274 294 313 189 285 291 256 320 221 261 299 266 201 307 224 208 209 242 186 198 205 241 214 302 173 322 199 178 228 232 254 325 179 310 258 270 298]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[46 157 14 106 7 43 88 158 25 39 56 65 67 148 153 27 79 122 0 44 71 93 124 168 24 33 105 138 155 29 115 131 146 22 83 166 45 87 92 59 95 132 134 145 19 78 85 99 4 121 167 10 142 74 116 12 41 54 64 81 113 150 170 1 34 141 2 140 161 20 50 86 136 38 144 40 82 160 112 126 130 77 15 32 48 133 156 80 108 6 70 135 109 3 61 75 139 143 52 129 31 60 118 152 163 35 102 119 162 16 101 117 123 13 30 51 57 91 73 128 72 76 96 62 127 5 11 26 154 9 97 49 55 66 68 63 104 23 114 169 8 28 42 94 36 90 147 165 53 89 100 103 111 125 149 137 17 151 21 47 159 69 120 18 58 84 107 110 37 98 164] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[245 283 222 239 262 263 275 323 333 231 286 305 202 246 312 253 226 243 257 249 219 288 318 213 337 177 216 324 197 295 317 330 223 235 314 327 259 303 301 172 260 278 191 255 267 276 292 194 203 338 180 210 304 190 196 271 319 273 340 174 182 175 236 280 289 287 185 229 328 277 195 279 192 237 240 326 206 300 315 176 282 329 336 200 290 296 204 227 252 281 233 234 218 265 297 339 183 274 294 313 189 285 291 256 320 221 261 299 266 201 307 224 208 209 242 205 186 198 214 241 302 173 322 199 254 178 228 232 325 179 310 258 270 298 184 188 215 308 309 335 193 207 225 272 293 247 251 334 211 212 238 217 264 316 331 220 230 181 187 250 171 268 321 284 306 332 244 248 269 311]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 log_level:Info active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:31:16.587-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047...
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:16.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:31:16.641-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:16.641-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:31:16.642-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:31:16.642-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:31:16.642-05:00 [INFO] Checkpointing starts in 1801.403 sec
PipelineRuntimeCtx 2017-04-25T13:31:16.642-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:31:16.642-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:31:16.642-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:31:16.642-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:31:16.643-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:31:16.643-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:31:16.643-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:31:16.643-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:31:16.643-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:31:16.674-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:31:16.683-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:31:16.683-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:31:16.683-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:31:16.703-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:31:16.703-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:31:16.706-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:16.706-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:16.706-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:31:16.707-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:31:16.713-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:31:16.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:31:16.729-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:16.729-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:31:16.730-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:31:16.730-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:31:16.730-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:31:16.769-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:31:16.769-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:31:16.769-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:31:16.807-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:31:16.807-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:31:16.873-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:31:16.933-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:31:16.933-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:16.933-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:31:16.933-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:31:16.933-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:31:16.933-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:31:16.933-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047 is started----------
PipelineManager 2017-04-25T13:31:16.934-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:31:16.934-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:31:16.934-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:31:17.463-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:31:17.464-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:31:17.464-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:31:17.464-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:31:17 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 103 successful
CapiNozzle 2017-04-25T13:31:17.604-05:00 [INFO] Send batch count=28
CapiNozzle 2017-04-25T13:31:17.605-05:00 [INFO] Send batch count=56
CapiNozzle 2017-04-25T13:31:17.619-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:17.619-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:17.630-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:31:17 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 7 successful
CapiNozzle 2017-04-25T13:31:17.655-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:17.655-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:17.668-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:31:17.682-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:17.705-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:31:17 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 307 successful
CapiNozzle 2017-04-25T13:31:17.737-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:31:17 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 213 successful
StatisticsManager 2017-04-25T13:31:17.761-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:31:17.768-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:17.814-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:31:17.819-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:31:17 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 28 successful
CapiNozzle 2017-04-25T13:31:17.870-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:17.870-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:31:17.870-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:31:17.870-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:31:17.870-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:31:17.886-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:31:17.886-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:31:17.889-05:00 [INFO] err_list=[{"time":"2017-04-25T13:31:17.889866545-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:31:17.890-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:31:17.890-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:17.921-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:31:17 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:31:17 UPR_STREAMREQ for vb 334 successful
CapiNozzle 2017-04-25T13:31:17.969-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:17.969-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:17.994-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:31:18.014-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:18.015-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:31:18.015-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:31:18.015-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:18.015-05:00 [INFO] Send batch count=500
2017/04/25 13:31:18 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 240 successful
CapiNozzle 2017-04-25T13:31:18.097-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:18.097-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:18.131-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
2017/04/25 13:31:18 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 153 successful
CapiNozzle 2017-04-25T13:31:18.935-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:18.953-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
StatisticsManager 2017-04-25T13:31:18.958-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:31:18.962-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047 {"Errors": "[{\"time\":\"2017-04-25T13:31:17.889866545-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 36, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 2567, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 36, "set_filtered": 0, "set_received_from_dcp": 2567, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

2017/04/25 13:31:18 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:31:18 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 67 successful
CapiNozzle 2017-04-25T13:31:19.318-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:19.342-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
2017/04/25 13:31:19 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 72 successful
CapiNozzle 2017-04-25T13:31:19.534-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:31:19 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 279 successful
CapiNozzle 2017-04-25T13:31:19.550-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:31:19 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:31:19.807-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:19.824-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
2017/04/25 13:31:19 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:31:19 UPR_STREAMREQ for vb 299 successful
CheckpointManager 2017-04-25T13:31:19.856-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
CheckpointManager 2017-04-25T13:31:19.856-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:31:19.938-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:31:19 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:31:19.950-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:19.950-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:31:19.950-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:31:19.950-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:19.950-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:20.033-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:20.034-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
DcpNozzle 2017-04-25T13:31:20.034-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:31:20.046-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:31:20.246-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:20.246-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:31:20.246-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:31:20.246-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:20.246-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:20.347-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:20.347-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:20.365-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:31:20.558-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:20.575-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:31:20.919-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:20.932-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:31:21.135-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:21.150-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:31:21.412-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:21.431-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:31:21.749-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:21.749-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:31:21.749-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:31:21.749-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:21.749-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:21.817-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:21.817-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:21.839-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:31:22.088-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:22.089-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:31:22.089-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:31:22.089-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:22.089-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:22.148-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:22.148-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:22.159-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:31:22.266-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:22.290-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:31:22.509-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:22.526-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:31:22.835-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:22.847-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:31:22.960-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:22.981-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:31:23.236-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:23.236-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:31:23.236-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:31:23.236-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:23.236-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:23.293-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:23.293-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:23.312-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:31:23.385-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:23.385-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:31:23.385-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:31:23.385-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:23.385-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:23.559-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:23.560-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:23.579-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:31:23.859-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:23.879-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:31:23.942-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:23.966-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:31:24.225-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:24.238-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:31:24.471-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:24.498-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:31:24.674-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:24.674-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:31:24.674-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:31:24.674-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:24.674-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:24.770-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:24.770-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:24.796-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:31:24.898-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:24.898-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:31:24.898-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:31:24.898-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:24.898-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:24.978-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:24.978-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:25.004-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:31:25.415-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:25.441-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:31:25.793-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:25.816-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:31:26.071-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:26.099-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:31:26.279-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:26.289-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:31:26.547-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:26.548-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:31:26.548-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:31:26.548-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:26.548-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:31:26.642-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:31:26.642-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:31:26.642-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:31:26.646-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:26.646-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:26.657-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:31:26.797-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:26.797-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:31:26.797-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:31:26.797-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:26.797-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:31:26.870-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:31:26.891-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:26.891-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:26.902-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:31:27.074-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:31:27.085-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:31:27.260-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:31:27.270-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:31:27.425-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:31:27.437-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
PipelineManager 2017-04-25T13:31:27.890-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:31:27.890-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:31:27.890-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:27.890-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
PipelineRuntimeCtx 2017-04-25T13:31:27.890-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:31:27.890-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:27.890-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:31:27.890-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:27.890408748 -0500 CDT
GenericSupervisor 2017-04-25T13:31:27.890-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:27.890-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:31:27.890-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:27.890-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:31:27.890-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:27.890-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:31:27.890-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:31:27.890-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:31:27.890-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:31:27.890-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-490076047 has been stopped
GenericPipeline 2017-04-25T13:31:27.890-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-490076047 is stopped
PipelineManager 2017-04-25T13:31:27.891-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:31:27.891-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:17.889866545-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:31:27.891-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:27.891-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
GenericPipeline 2017-04-25T13:31:27.891-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
GenericPipeline 2017-04-25T13:31:27.891-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
GenericPipeline 2017-04-25T13:31:27.891-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-490076047
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:27.891-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.891-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:27.892-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:31:27.892-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:31:27.892-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:27.892-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:31:27.892-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:27.892-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:31:27.892-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:31:27.892-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:31:27.892-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:27.892-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:31:27.892-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:31:27.892-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:27.892038563 -0500 CDT
AsyncListener 2017-04-25T13:31:27.892-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:27.892-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:27.892615483 -0500 CDT
DcpNozzle 2017-04-25T13:31:27.892-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:31:27.893-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:31:27.893-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:27.893-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:31:27.893-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:27.894-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:31:27.894-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:31:27.894-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:31:27.894-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:27.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:31:27.894-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:27.894-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:31:27.894-05:00 [INFO] Finish executing 0x65e930
DcpNozzle 2017-04-25T13:31:27.894-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:31:27.894-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:31:27.894-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:27.894399891 -0500 CDT
DcpNozzle 2017-04-25T13:31:27.894-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:31:27.894-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:27.894-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:31:27.896-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:27.896-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:27.896520154 -0500 CDT
DcpNozzle 2017-04-25T13:31:27.896-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:31:27.896-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:27.896-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:31:27 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:31:27 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:31:28.008-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:28.008-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:31:28.008-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:31:28.052-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:28.053-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:31:28.053-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:28.053-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:31:28.053-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:28.053-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:31:28.053-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:31:28.054-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:31:28.054-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
GenericSupervisor 2017-04-25T13:31:28.056-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:31:28.056-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
GenericSupervisor 2017-04-25T13:31:28.056-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:28.056-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:28.056-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:28.056-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:28.056-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:31:28.056-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
CheckpointManager 2017-04-25T13:31:28.056-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
CapiUtils 2017-04-25T13:31:28.085-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:28.085-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:28.085-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:31:28.086-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
StatisticsManager 2017-04-25T13:31:28.086-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:31:28.086-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
XDCRFactory 2017-04-25T13:31:28.086-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:31:28.086-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551 is constructed, start it
GenericPipeline 2017-04-25T13:31:28.088-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[47 44 113 116 111 36 91 136 162 14 73 127 66 152 123 53 121 24 39 79 22 84 29 89 119 163 168 9 133 166 167 78 146 25 100 18 54 96 159 21 60 63 122 27 5 85 76 17 45 59 62 0 13 92 148 3 101 120 151 49 140 82 107 157 23 142 144 169 15 104 90 33 35 4 98 134 61 34 72 32 117 86 41 10 19 70 115 126 135 139 2 160 125 71 56 87 114 124 129 164 11 74 77 106 109 118 137 37 65 147 7 40 99 31 69 50 149 46 75 150 28 170 80 88 12 138 38 20 112 52 58 128 145 55 110 158 97 102 156 48 64 131 83 94 143 51 68 154 67 30 105 26 57 16 81 161 43 108 130 132 165 95 6 93 141 153 155 1 42 103 8] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[198 230 234 302 193 196 228 244 323 221 253 261 336 338 231 214 259 262 290 307 209 200 328 176 314 334 318 242 330 206 249 329 191 236 240 286 295 298 222 284 311 279 280 301 248 247 294 326 327 177 224 226 270 283 306 225 272 258 265 312 211 299 190 199 269 315 201 292 207 332 333 174 215 237 297 316 185 238 255 304 322 246 313 308 189 202 340 203 213 275 278 291 319 339 186 276 219 324 325 172 260 285 293 175 194 287 188 197 305 250 205 187 216 171 184 257 289 179 180 181 183 223 263 277 331 320 173 217 229 252 321 281 310 337 182 335 220 271 288 204 303 192 241 233 245 266 264 268 212 227 256 317 251 267 210 208 218 254 274 232 235 243 300 273 282 195 296 178 239 309]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[340 189 202 278 291 319 339 203 213 275 186 219 276 324 325 260 285 293 172 194 287 175 188 197 305 205 250 187 216 289 171 184 257 183 223 263 277 179 180 181 320 331 173 252 321 217 229 281 310 337 182 335 220 271 288 303 204 241 192 233 245 266 264 268 317 212 227 256 210 251 267 254 274 208 218 235 243 300 232 273 282 195 296 239 309 178 198 230 302 234 193 196 228 244 323 261 336 338 221 253 231 259 262 290 214 209 307 200 328 176 314 318 334 242 330 206 249 329 236 240 286 295 191 298 222 284 311 279 280 301 248 327 247 294 326 177 283 224 226 270 306 225 272 258 265 312 211 299 190 199 269 315 201 292 207 333 332 215 237 297 316 174 255 304 322 185 238 246 308 313] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[125 160 71 124 129 164 11 56 87 114 109 118 137 37 74 77 106 7 65 147 31 40 99 69 46 50 149 28 75 150 170 80 12 88 38 138 20 52 112 55 58 128 145 97 110 158 48 102 156 64 83 131 51 94 143 67 68 154 30 26 105 16 57 43 81 161 165 95 108 130 132 153 155 1 6 93 141 8 42 103 44 47 111 113 116 162 14 36 91 136 66 73 127 152 53 123 24 121 22 39 79 84 163 168 9 29 89 119 78 133 166 167 146 18 25 100 21 54 96 159 27 60 63 122 5 85 76 62 0 17 45 59 13 3 92 148 49 101 120 151 140 23 82 107 157 15 142 144 169 90 104 4 33 35 61 98 134 32 34 72 86 117 10 41 126 135 139 2 19 70 115]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[filter_expression: active:true failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 replication_type:capi worker_batch_size:500 doc_batch_size_kb:2048 target_nozzle_per_node:2 log_level:Info checkpoint_interval:1800]
CheckpointManager 2017-04-25T13:31:28.088-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551...
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:28.097-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:31:28.097-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:31:28.097-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:31:28.097-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:28.097-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:31:28.097-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:31:28.097-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:31:28.097-05:00 [INFO] Checkpointing starts in 1803.866 sec
PipelineRuntimeCtx 2017-04-25T13:31:28.097-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:31:28.097-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:31:28.098-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:31:28.098-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:31:28.102-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:31:28.102-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:31:28.106-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:31:28.109-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:31:28.133-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:31:28.133-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:31:28.133-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:31:28.145-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:28.145-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:28.145-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:31:28.159-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:31:28.159-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:31:28.187-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:31:28.237-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:31:28.263-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:31:28.263-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:28.263-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:31:28.264-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:31:28.264-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:31:28.264-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:31:28.293-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:31:28.307-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:31:28.307-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:31:28.307-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:31:28.318-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:31:28.318-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:31:28.455-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:31:28.455-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:28.455-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:31:28.455-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:31:28.455-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:31:28.455-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:31:28.456-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551 is started----------
PipelineManager 2017-04-25T13:31:28.456-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:31:28.456-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:31:28.456-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:31:28.554-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:31:28.555-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:31:28.555-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:31:28.555-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:31:28 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:31:28.718-05:00 [INFO] Send batch count=45
CapiNozzle 2017-04-25T13:31:28.734-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:28.735-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:28 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:31:28.777-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:31:28 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:31:28.792-05:00 [INFO] Send batch count=124
CapiNozzle 2017-04-25T13:31:28.835-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:28.835-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:28.854-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
2017/04/25 13:31:28 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:31:28 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 319 successful
CapiNozzle 2017-04-25T13:31:29.136-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:29.174-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
StatisticsManager 2017-04-25T13:31:29.186-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:31:29 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 35 successful
CapiNozzle 2017-04-25T13:31:29.315-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:31:29.329-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:29.330-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:31:29.352-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
2017/04/25 13:31:29 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 50 successful
CapiNozzle 2017-04-25T13:31:29.449-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:29.449-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:31:29.449-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:31:29.449-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
ReplicationManager 2017-04-25T13:31:29.450-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
PipelineManager 2017-04-25T13:31:29.468-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:31:29.468-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:31:29.470-05:00 [INFO] err_list=[{"time":"2017-04-25T13:31:29.470836544-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]
PipelineManager 2017-04-25T13:31:29.470-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:31:29 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 155 successful
CapiNozzle 2017-04-25T13:31:29.558-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:29.558-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:29.578-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:31:29 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 269 successful
CapiNozzle 2017-04-25T13:31:29.610-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:31:29.630-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
2017/04/25 13:31:29 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 93 successful
CheckpointManager 2017-04-25T13:31:29.876-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
CheckpointManager 2017-04-25T13:31:29.876-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:31:29 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:31:29 UPR_STREAMREQ for vb 293 successful
DcpNozzle 2017-04-25T13:31:29.956-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:31:29.963-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:29.986-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:31:29 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:31:30.068-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:31:30.095-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:31:30.095-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:31:30.095-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:31:30.095-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:30.095-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:31:30.104-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:31:30.104-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551 {"Errors": "[{\"time\":\"2017-04-25T13:31:29.470836544-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 58, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 18403, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 58, "set_filtered": 0, "set_received_from_dcp": 18403, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:31:30.206-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:30.207-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:30.229-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:31:30.457-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:31.507-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:31:31.518-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:31.563-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
PipelineManager 2017-04-25T13:31:31.756-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:29.470836544-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]}
]
CapiNozzle 2017-04-25T13:31:31.908-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:31:31.908-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:31:31.908-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:31:31.908-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:31.908-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:31.993-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:31.993-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:32.007-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
ReplicationManager 2017-04-25T13:31:32.025-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:31:32.265-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:32.289-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:31:32.381-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:32.401-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:31:32.723-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:32.723-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:31:32.723-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:31:32.723-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:32.723-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:32.777-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:32.777-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:32.796-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:31:32.866-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:32.885-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:31:33.121-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:33.143-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:31:33.415-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:31:33.415-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:31:33.415-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:31:33.415-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:33.415-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:33.534-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:33.534-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:33.570-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:31:33.636-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:33.649-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:31:33.978-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:31:34.006-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:31:34.152-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:34.152-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:31:34.152-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:31:34.152-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:34.152-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:34.224-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:34.224-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:34.245-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:31:34.327-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:31:34.342-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:31:34.641-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:34.657-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:31:34.767-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:31:34.767-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:31:34.767-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:31:34.767-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:34.767-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:34.833-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:34.833-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:34.853-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:31:34.964-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:34.976-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:31:35.131-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:35.147-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:31:35.282-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:35.282-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:31:35.282-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:31:35.282-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:35.282-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:35.378-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:35.378-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:35.394-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:31:35.451-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:35.479-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:31:36.027-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:36.045-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:31:36.301-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:36.301-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:31:36.301-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:31:36.301-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:36.301-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:36.419-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:36.419-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:36.444-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:31:36.720-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:36.742-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:31:36.872-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:31:36.884-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:31:37.231-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:37.231-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:31:37.231-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:31:37.231-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:37.231-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:37.321-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:37.321-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:37.335-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:31:37.356-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:31:37.368-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:31:37.794-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:37.806-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:31:37.939-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:31:37.939-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:31:37.939-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:31:37.939-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:37.939-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:38.067-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:38.067-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:38.078-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
GenericSupervisor 2017-04-25T13:31:38.098-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:31:38.098-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:31:38.099-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:31:38.284-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
ToplogyChangeDetector 2017-04-25T13:31:38.288-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:31:38.294-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:31:38.488-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:38.499-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:31:38.627-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:38.627-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:31:38.627-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:31:38.627-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:38.627-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:38.727-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:38.728-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:38.739-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:31:38.955-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:38.966-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:31:39.235-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:39.246-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
PipelineManager 2017-04-25T13:31:39.471-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries 
PipelineManager 2017-04-25T13:31:39.471-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:31:39.471-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:39.471-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
PipelineRuntimeCtx 2017-04-25T13:31:39.471-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:31:39.471-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:39.471-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:31:39.471-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:39.47139838 -0500 CDT
GenericSupervisor 2017-04-25T13:31:39.471-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:39.471-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:31:39.471-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:39.471-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:31:39.471-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:39.471-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:31:39.471-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:31:39.471-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:31:39.471-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:31:39.471-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-53936551 has been stopped
GenericPipeline 2017-04-25T13:31:39.472-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-53936551 is stopped
PipelineManager 2017-04-25T13:31:39.472-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:31:39.472-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:29.470836544-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:31:39.472-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:39.472-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
GenericPipeline 2017-04-25T13:31:39.472-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:31:39.472-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:31:39.472-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-53936551
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:31:39.472-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.472-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:31:39.472-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:39.473-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:31:39.473-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:39.473-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:31:39.473-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:31:39.473-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:39.473-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:39.473-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:31:39.473-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:39.473-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:39.473-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:31:39.473-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:39.473291704 -0500 CDT
CapiNozzle 2017-04-25T13:31:39.473-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:39.473408013 -0500 CDT
CapiNozzle 2017-04-25T13:31:39.474-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:31:39.474-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:39.474-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:31:39.474-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:31:39.474-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:31:39.474-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:31:39.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:31:39.474-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:31:39.474-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:39.474-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:31:39.476-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:39.476-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:39.476338001 -0500 CDT
DcpNozzle 2017-04-25T13:31:39.476-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:39.476-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:31:39.476-05:00 [INFO] Finish executing 0x65e930
DcpNozzle 2017-04-25T13:31:39.476-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:39.4763891 -0500 CDT
GenericPipeline 2017-04-25T13:31:39.476-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:31:39.476-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:31:39.476-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:39.476-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:31:39 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:31:39 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:31:39 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:31:39 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:31:39 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:31:39 Stream Closed for vb 175 StreamEnd simulated
XDCRFactory 2017-04-25T13:31:39.610-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:39.610-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:31:39.611-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:31:39.817-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:39.818-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:31:39.818-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:39.818-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:31:39.818-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:39.818-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:31:39.818-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:31:39.819-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:31:39.819-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
GenericSupervisor 2017-04-25T13:31:39.821-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:31:39.821-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
GenericSupervisor 2017-04-25T13:31:39.822-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:39.822-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:39.822-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:39.822-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:39.822-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:31:39.822-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
CheckpointManager 2017-04-25T13:31:39.822-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
CapiUtils 2017-04-25T13:31:39.875-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:39.875-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:39.875-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:31:39.876-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
StatisticsManager 2017-04-25T13:31:39.877-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:31:39.877-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
XDCRFactory 2017-04-25T13:31:39.877-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:31:39.877-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247 is constructed, start it
GenericPipeline 2017-04-25T13:31:39.878-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[280 327 339 194 271 191 298 340 171 264 211 226 220 282 279 228 269 246 305 238 275 310 199 237 240 178 203 233 235 307 320 216 333 202 258 186 187 196 180 277 281 175 224 229 247 302 323 219 227 234 262 288 201 249 252 314 197 243 336 297 334 245 293 322 172 200 176 225 259 231 236 317 213 217 221 335 183 273 294 299 210 261 331 189 318 313 329 173 184 205 230 303 188 289 260 274 223 292 309 324 214 286 338 222 306 319 179 209 296 284 311 244 267 270 198 316 332 326 337 185 192 174 212 218 312 257 254 265 268 290 325 241 304 177 182 295 255 190 204 181 242 287 206 207 215 315 300 250 276 251 208 272 321 285 291 193 195 328 239 263 283 253 256 278 248 301 308 266 330 232] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[11 55 120 43 79 30 141 72 94 147 23 139 78 66 81 128 169 12 45 134 143 42 67 168 170 32 35 124 17 25 105 97 113 9 18 65 132 8 129 73 108 160 77 148 3 24 107 99 119 149 13 20 110 117 63 10 21 33 47 116 150 166 68 70 85 140 1 36 58 152 86 162 64 101 125 131 135 6 46 54 62 98 142 159 100 122 161 4 28 50 53 165 51 104 145 0 7 44 111 16 19 57 83 92 123 137 163 15 27 40 41 118 136 153 26 164 82 91 31 52 155 96 34 37 75 154 14 89 144 39 60 112 59 102 106 146 61 5 88 130 48 114 121 76 156 2 38 103 49 56 151 167 133 157 87 90 93 138 29 95 126 127 84 109 115 158 22 69 71 74 80]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[175 224 229 247 302 323 219 227 262 288 234 249 201 197 252 314 243 336 297 334 245 293 322 172 200 176 225 259 213 217 221 231 236 317 335 183 273 294 299 210 261 189 318 331 313 329 173 184 205 230 303 188 289 223 260 274 214 286 292 309 324 338 222 179 209 296 306 319 244 267 270 284 311 198 316 332 185 192 326 337 174 212 218 312 254 257 268 290 325 265 177 182 241 304 295 190 204 255 181 242 287 206 207 215 315 250 276 300 251 208 272 321 193 195 285 291 328 239 263 283 248 253 256 278 308 301 266 330 232 194 271 280 327 339 191 298 340 171 264 211 226 220 282 228 279 269 238 246 305 199 237 275 310 178 203 233 240 216 235 307 320 333 186 187 196 202 258 180 277 281] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[3 107 24 119 149 99 20 110 117 13 63 21 33 10 116 150 47 166 70 85 140 68 36 58 1 152 162 86 101 125 131 135 64 46 54 62 98 6 159 142 122 161 100 28 50 53 165 4 104 145 51 0 44 111 7 19 57 83 16 123 137 92 27 40 41 118 163 15 153 136 164 26 91 82 52 155 31 96 37 75 154 34 89 144 14 60 112 39 102 106 59 146 61 88 130 5 114 121 48 156 76 38 103 2 56 151 167 49 157 133 90 87 138 93 95 126 127 29 109 115 84 69 71 74 80 158 22 55 120 11 79 43 141 30 94 147 72 139 23 78 81 128 169 66 45 134 143 12 67 168 170 42 35 124 32 25 105 17 113 97 18 65 132 9 129 8 108 160 73 148 77]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info filter_expression: checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 replication_type:capi active:true optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:31:39.878-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247...
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:39.898-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:39.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:39.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:31:39.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
StatisticsManager 2017-04-25T13:31:39.899-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:31:39.901-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:31:39.901-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:31:39.901-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:31:39.901-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:39.901-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:31:39.901-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:31:39.901-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:31:39.902-05:00 [INFO] Checkpointing starts in 1801.809 sec
PipelineRuntimeCtx 2017-04-25T13:31:39.902-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:31:39.902-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:31:39.903-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:31:39.903-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:31:39.905-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:31:39.916-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:39.916-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:39.916-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:31:39.934-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:31:39.935-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:31:39.935-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
ToplogyChangeDetector 2017-04-25T13:31:39.948-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:31:39.948-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:31:39.954-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:31:39.954-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:31:39.977-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:31:39.991-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:31:39.991-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:39.991-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:31:39.991-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:31:39.991-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:31:39.991-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:31:40.048-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:31:40.048-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:31:40.058-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:31:40.088-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:31:40.088-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:31:40.140-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:31:40.329-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:31:40.329-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:31:40.330-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:31:40.330-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:31:40.337-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:31:40.337-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:40.337-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:31:40.337-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:31:40.337-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:31:40.337-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:31:40.337-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247 is started----------
PipelineManager 2017-04-25T13:31:40.337-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:31:40.337-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:31:40.337-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:31:40 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:31:40.456-05:00 [INFO] Send batch count=74
2017/04/25 13:31:40 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:31:40.497-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:31:40.513-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:40.513-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:40 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:31:40.540-05:00 [INFO] Send batch count=188
CapiNozzle 2017-04-25T13:31:40.542-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:31:40 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 209 successful
CapiNozzle 2017-04-25T13:31:40.654-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:40.654-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:40 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 10 successful
CapiNozzle 2017-04-25T13:31:40.670-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
2017/04/25 13:31:40 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 310 successful
CapiNozzle 2017-04-25T13:31:40.713-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:40.764-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
2017/04/25 13:31:40 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:31:40 UPR_STREAMREQ for vb 27 successful
CapiNozzle 2017-04-25T13:31:40.974-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:31:40.993-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
StatisticsManager 2017-04-25T13:31:40.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:31:41 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 335 successful
CapiNozzle 2017-04-25T13:31:41.040-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:31:41 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 142 successful
CapiNozzle 2017-04-25T13:31:41.065-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:31:41 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 54 successful
CapiNozzle 2017-04-25T13:31:41.259-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:31:41.291-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
2017/04/25 13:31:41 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 259 successful
CapiNozzle 2017-04-25T13:31:41.325-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:41.325-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:31:41.325-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:31:41.325-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
ReplicationManager 2017-04-25T13:31:41.325-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
PipelineManager 2017-04-25T13:31:41.342-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:31:41.342-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:31:41.345-05:00 [INFO] err_list=[{"time":"2017-04-25T13:31:41.345660337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]
PipelineManager 2017-04-25T13:31:41.345-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:31:41 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:31:41.447-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:41.447-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:41 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 170 successful
CapiNozzle 2017-04-25T13:31:41.472-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:31:41 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 85 successful
CapiNozzle 2017-04-25T13:31:41.556-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:31:41.556-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:31:41.556-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:31:41.556-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:41.556-05:00 [INFO] Send batch count=500
2017/04/25 13:31:41 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 189 successful
CapiNozzle 2017-04-25T13:31:41.655-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:41.655-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:41 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 92 successful
CapiNozzle 2017-04-25T13:31:41.688-05:00 [INFO] Retrying update docs for vb 106 for the 1th time
2017/04/25 13:31:41 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 198 successful
CheckpointManager 2017-04-25T13:31:41.719-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
CheckpointManager 2017-04-25T13:31:41.719-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:31:41 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:31:41 UPR_STREAMREQ for vb 297 successful
DcpNozzle 2017-04-25T13:31:41.844-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:31:41.876-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:41.912-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
DcpNozzle 2017-04-25T13:31:41.922-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:31:41.922-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:31:41.923-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247 {"Errors": "[{\"time\":\"2017-04-25T13:31:41.345660337-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 18, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 113, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 18402, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 113, "set_filtered": 0, "set_received_from_dcp": 18402, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:31:42.117-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:31:42.153-05:00 [INFO] Retrying update docs for vb 106 for the 2th time
CapiNozzle 2017-04-25T13:31:42.425-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:42.447-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:31:42.623-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:31:42.667-05:00 [INFO] Retrying update docs for vb 106 for the 3th time
CapiNozzle 2017-04-25T13:31:43.153-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:31:43.154-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:31:43.154-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:31:43.154-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:43.154-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:43.281-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:43.283-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:43.298-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:31:43.372-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:31:43.372-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 106 after 3 retries
CapiNozzle 2017-04-25T13:31:43.372-05:00 [ERROR] Raise error condition batch update docs failed for vb 106 after 3 retries
GenericSupervisor 2017-04-25T13:31:43.372-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:43.372-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:43.484-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:43.484-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:43.502-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:31:43.738-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:43.760-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:31:43.856-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:31:43.870-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:31:44.230-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:44.248-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:31:44.406-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:31:44.443-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:31:44.756-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:31:44.756-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:31:44.756-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:31:44.756-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:44.756-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:44.865-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:44.866-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:44.886-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:31:45.204-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:31:45.222-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:31:45.222-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:31:45.222-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:45.222-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:45.304-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:45.305-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:45.335-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:31:45.598-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:45.632-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
PipelineManager 2017-04-25T13:31:45.859-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:41.345660337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]}
]
CapiNozzle 2017-04-25T13:31:46.084-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:46.113-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:31:46.317-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:46.337-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:31:46.591-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:46.617-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:31:47.081-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:31:47.081-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:31:47.081-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:31:47.081-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:47.081-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:47.140-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:47.140-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:47.162-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:31:47.507-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:31:47.507-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:31:47.507-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:31:47.507-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:47.507-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:47.586-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:47.586-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:47.604-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:31:47.660-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:47.683-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:31:47.985-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:48.003-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:31:48.097-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:48.121-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:31:48.297-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:48.309-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:31:48.433-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:48.433-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:31:48.433-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:31:48.433-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:48.433-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:48.554-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:48.554-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:48.568-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:31:48.619-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:48.619-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:31:48.619-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:31:48.619-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:48.619-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:48.748-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:48.749-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:48.762-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:31:48.822-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:48.835-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:31:49.304-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:31:49.315-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:31:49.584-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:49.598-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
GenericSupervisor 2017-04-25T13:31:49.904-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:31:49.904-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:31:49.904-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:31:50.016-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:31:50.027-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
ToplogyChangeDetector 2017-04-25T13:31:50.211-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:31:50.347-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:31:50.347-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:31:50.347-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:31:50.347-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:50.347-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:50.432-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:50.432-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:50.443-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:31:50.487-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:31:50.487-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:31:50.487-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:31:50.487-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:50.487-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:50.558-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:50.558-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:50.569-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
CapiNozzle 2017-04-25T13:31:50.891-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:50.901-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:31:51.024-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:31:51.035-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
PipelineManager 2017-04-25T13:31:53.360-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries 
PipelineManager 2017-04-25T13:31:53.360-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:31:53.360-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:53.360-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
PipelineRuntimeCtx 2017-04-25T13:31:53.360-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:31:53.360-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:31:53.360-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:31:53.360-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:31:53.360-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:53.360-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:31:53.360-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:53.360415021 -0500 CDT
GenericSupervisor 2017-04-25T13:31:53.360-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:53.360-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:31:53.361-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:53.361-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:31:53.361-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:31:53.361-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:31:53.361-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-818971247 has been stopped
GenericPipeline 2017-04-25T13:31:53.361-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-818971247 is stopped
PipelineManager 2017-04-25T13:31:53.361-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:31:53.361-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:41.345660337-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:31:53.361-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:31:53.361-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
GenericPipeline 2017-04-25T13:31:53.361-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
GenericPipeline 2017-04-25T13:31:53.361-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
GenericPipeline 2017-04-25T13:31:53.362-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-818971247
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:31:53.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:31:53.362-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:53.362-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:31:53.362-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:31:53.362-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:53.362-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:31:53.362-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:53.363-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:31:53.363-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:31:53.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:31:53.363-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:31:53.363-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:31:53.363-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:53.362919335 -0500 CDT
DcpNozzle 2017-04-25T13:31:53.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:31:53.363-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:53.363102005 -0500 CDT
DcpNozzle 2017-04-25T13:31:53.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:31:53.365-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:31:53.365-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:53.365-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:31:53.365-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:31:53.365-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:31:53.365-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:53.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:31:53.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:31:53.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:31:53.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:31:53.366-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:53.366-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:31:53.366-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:53.366-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:31:53.370-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:53.370-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:53.370562038 -0500 CDT
DcpNozzle 2017-04-25T13:31:53.370-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:31:53.370-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:53.370-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:31:53 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:31:53.381-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:31:53.382-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:31:53.381981272 -0500 CDT
DcpNozzle 2017-04-25T13:31:53.382-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:31:53.382-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:31:53.382-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:31:53 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:31:54.324-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:54.324-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:31:54.324-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:31:54.467-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:31:54.467-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:31:54.467-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:54.467-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:31:54.467-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:31:54.467-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:31:54.467-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:31:54.468-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:31:54.468-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
GenericSupervisor 2017-04-25T13:31:54.469-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:31:54.469-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
GenericSupervisor 2017-04-25T13:31:54.469-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:54.469-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:54.469-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:54.469-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:31:54.469-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:31:54.469-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
CheckpointManager 2017-04-25T13:31:54.469-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
CapiUtils 2017-04-25T13:31:54.502-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:54.502-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:54.502-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:31:54.502-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
StatisticsManager 2017-04-25T13:31:54.503-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:31:54.503-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
XDCRFactory 2017-04-25T13:31:54.503-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:31:54.503-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019 is constructed, start it
GenericPipeline 2017-04-25T13:31:54.503-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[13 23 120 161 11 66 69 41 131 139 147 149 37 91 38 109 19 146 63 97 119 132 47 70 21 72 137 169 83 105 136 44 102 113 8 18 45 76 95 25 53 155 52 61 24 74 96 108 126 158 88 134 148 164 46 62 71 14 165 124 152 28 86 118 122 3 31 50 65 75 117 159 58 73 135 167 16 84 30 34 48 94 49 90 92 141 22 93 99 57 128 151 68 9 36 54 79 157 111 123 125 143 6 4 27 29 78 166 17 56 107 133 33 81 101 130 142 170 116 154 127 140 153 7 15 35 51 144 145 85 89 138 162 0 12 106 26 32 129 60 1 64 87 20 114 39 40 115 2 80 110 121 156 82 43 77 104 150 5 100 112 10 42 55 59 67 98 160 168 103 163] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[189 219 240 243 310 271 237 261 269 326 196 203 267 309 215 314 327 213 172 194 235 265 281 173 286 236 259 191 276 323 329 308 311 181 256 319 174 214 234 295 328 244 193 200 231 264 274 304 339 220 222 273 331 280 289 305 315 184 209 217 224 270 300 297 306 212 255 284 316 253 185 188 275 178 186 245 211 268 180 226 299 322 179 227 230 248 263 272 199 239 282 266 334 233 313 325 176 195 238 307 202 218 303 336 247 279 242 277 333 198 317 330 223 241 318 321 225 207 177 228 250 251 197 208 257 182 249 262 171 204 221 260 332 190 283 337 291 338 340 205 206 285 298 301 335 232 278 287 254 290 175 183 294 296 192 201 210 258 288 229 292 312 320 216 293 187 246 252 302 324]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[334 266 233 313 325 176 195 238 307 303 336 202 218 247 279 277 333 242 198 317 330 223 241 318 321 225 207 177 228 250 251 197 208 257 182 249 262 171 204 221 260 332 190 283 337 291 338 340 206 285 298 301 335 205 232 278 287 254 290 175 183 294 296 192 201 210 258 288 292 229 312 320 216 293 187 246 252 302 324 240 243 310 189 219 271 237 261 269 326 196 203 267 309 215 314 327 213 194 235 265 281 172 173 286 236 259 191 276 323 329 308 311 181 256 319 174 214 234 295 328 244 200 231 264 274 304 339 193 222 273 331 220 289 305 315 280 184 209 217 224 270 300 297 306 212 255 284 316 253 185 188 275 178 186 245 211 268 180 226 299 322 179 227 230 248 263 272 199 239 282] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[68 54 79 157 9 36 125 143 111 123 6 29 78 166 4 27 107 133 17 56 101 130 142 33 81 170 116 154 153 127 140 35 51 144 145 7 15 138 162 85 89 0 12 106 129 26 32 60 87 1 64 20 114 115 39 40 110 121 156 2 80 82 104 150 43 77 112 5 100 55 59 67 98 160 168 10 42 103 163 120 161 13 23 69 11 66 139 147 149 41 131 37 91 38 109 19 146 63 132 97 119 47 70 137 169 21 72 136 83 105 113 44 102 45 76 95 8 18 155 25 53 52 61 96 108 126 24 74 158 88 134 148 164 71 46 62 14 165 124 152 118 122 28 86 50 65 75 117 159 3 31 135 167 58 73 16 84 48 94 30 34 49 90 92 141 99 22 93 151 57 128]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 active:true doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:31:54.504-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019...
AsyncListener 2017-04-25T13:31:54.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:54.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:54.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:31:54.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
CheckpointManager 2017-04-25T13:31:54.511-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:31:54.511-05:00 [INFO] Checkpointing starts in 1804.541 sec
PipelineRuntimeCtx 2017-04-25T13:31:54.511-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:31:54.511-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:31:54.511-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:31:54.512-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:31:54.512-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:31:54.512-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:31:54.512-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:31:54.512-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:31:54.512-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:31:54.512-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:31:54.513-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:31:54.515-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:31:54.544-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:31:54.546-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:31:54.546-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:31:54.546-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:31:54.608-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:31:54.608-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:31:54.608-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:31:54.625-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:31:54.625-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:31:54.625-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:31:54.817-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:31:55.263-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:31:55.263-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:55.264-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:31:55.264-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:31:55.264-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:31:55.264-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:31:55.282-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:31:55.282-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:31:55.285-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:31:55.301-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:31:55.301-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:31:55.410-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:31:55.603-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:31:55.726-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:31:55.727-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:31:55.727-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:31:55.727-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:31:55.727-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:31:55.727-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:31:55.727-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019 is started----------
PipelineManager 2017-04-25T13:31:55.727-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:31:55.727-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:31:55.727-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:31:55.768-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:31:55.769-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:31:55.770-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:31:55.770-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:31:55 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:31:55.843-05:00 [INFO] Send batch count=317
2017/04/25 13:31:55 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:31:55.880-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:31:55.915-05:00 [INFO] Send batch count=365
CapiNozzle 2017-04-25T13:31:55.925-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:55.925-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:31:55 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:31:55 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:31:55.973-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:31:56 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 106 successful
CapiNozzle 2017-04-25T13:31:56.054-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:31:56 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 307 successful
CapiNozzle 2017-04-25T13:31:56.080-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:31:56 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 42 successful
CapiNozzle 2017-04-25T13:31:56.385-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:56.407-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:31:56 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 246 successful
StatisticsManager 2017-04-25T13:31:56.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:31:56.580-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:56.596-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:31:56 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 174 successful
CapiNozzle 2017-04-25T13:31:56.842-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:56.867-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:31:56 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:31:56 UPR_STREAMREQ for vb 275 successful
CapiNozzle 2017-04-25T13:31:57.029-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:31:57 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 192 successful
CapiNozzle 2017-04-25T13:31:57.057-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
2017/04/25 13:31:57 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:31:57.131-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
CheckpointManager 2017-04-25T13:31:57.131-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:31:57 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:31:57 UPR_STREAMREQ for vb 292 successful
DcpNozzle 2017-04-25T13:31:57.167-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:31:57.212-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:57.213-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:31:57.213-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:31:57.213-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:31:57.213-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:31:57.223-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:31:57.223-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:31:57.226-05:00 [INFO] err_list=[{"time":"2017-04-25T13:31:57.22640607-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:31:57.226-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
DcpNozzle 2017-04-25T13:31:57.230-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:31:57.296-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:57.297-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:57.318-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:31:57.367-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:31:57.367-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:31:57.367-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:31:57.367-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:57.368-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:57.464-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:57.464-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:57.479-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
StatisticsManager 2017-04-25T13:31:57.523-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:31:57.523-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019 {"Errors": "[{\"time\":\"2017-04-25T13:31:57.22640607-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 347, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 27236, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 347, "set_filtered": 0, "set_received_from_dcp": 27236, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:31:57.728-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:57.743-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:31:57.972-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:58.005-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:31:58.299-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:58.318-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:31:58.508-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:58.525-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:31:58.680-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:31:58.680-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:31:58.680-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:31:58.680-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:58.680-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:58.770-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:58.770-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:58.787-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:31:58.958-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:31:58.958-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:31:58.958-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:31:58.958-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:31:58.958-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:31:59.033-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:31:59.033-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:31:59.059-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:31:59.196-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:59.222-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:31:59.438-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:59.459-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:31:59.583-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:31:59.608-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:31:59.900-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:31:59.921-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:32:00.075-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:32:00.075-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:32:00.075-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:32:00.075-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:00.075-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:00.192-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:00.192-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:00.216-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:32:00.625-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:32:00.625-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:32:00.625-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:32:00.625-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:00.625-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:00.721-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:00.721-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:00.740-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
PipelineManager 2017-04-25T13:32:00.887-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:57.22640607-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:32:00.919-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:32:00.939-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:32:01.312-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:32:01.334-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:32:01.585-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:32:01.611-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:32:01.874-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:32:01.917-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:32:02.115-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:32:02.115-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:32:02.115-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:32:02.115-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:02.115-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:02.191-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:02.191-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:02.220-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:32:02.448-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:32:02.448-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:32:02.448-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:32:02.448-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:02.448-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:02.546-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:02.546-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:02.567-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:32:02.635-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:32:02.649-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:32:02.937-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:32:02.958-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:32:03.504-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:32:03.522-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:32:03.788-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:32:03.814-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:32:04.026-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:32:04.026-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:32:04.026-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:32:04.026-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:04.026-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:04.080-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:04.080-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:04.095-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:32:04.277-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:32:04.277-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:32:04.277-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:32:04.277-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:04.277-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:04.389-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:04.389-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:04.403-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:32:04.491-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:04.502-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
GenericSupervisor 2017-04-25T13:32:04.513-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:32:04.513-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:32:04.513-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:32:04.859-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:32:04.870-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:32:05.131-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:05.142-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:32:05.523-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:32:05.534-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
ToplogyChangeDetector 2017-04-25T13:32:05.620-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:32:05.771-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:05.771-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:32:05.771-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:32:05.771-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:05.771-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:05.863-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:05.863-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:05.874-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:32:06.070-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:32:06.070-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:32:06.070-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:32:06.070-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:06.070-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:06.191-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:06.191-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:06.202-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:32:06.329-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:32:06.340-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:32:06.600-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:32:06.611-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:32:06.849-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:32:06.860-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:32:07.066-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:32:07.077-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
PipelineManager 2017-04-25T13:32:07.226-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:32:07.226-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:32:07.226-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:32:07.226-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
PipelineRuntimeCtx 2017-04-25T13:32:07.226-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:32:07.226-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:07.226-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:32:07.226-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:07.226926662 -0500 CDT
GenericSupervisor 2017-04-25T13:32:07.227-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:07.227-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:32:07.227-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:32:07.227-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:32:07.227-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:32:07.227-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:32:07.227-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:32:07.227-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:32:07.227-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:32:07.227-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-467729019 has been stopped
GenericPipeline 2017-04-25T13:32:07.227-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-467729019 is stopped
PipelineManager 2017-04-25T13:32:07.227-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:32:07.227-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:31:57.22640607-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:32:07.227-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:32:07.227-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:32:07.227-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
GenericPipeline 2017-04-25T13:32:07.227-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
AsyncListener 2017-04-25T13:32:07.227-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:32:07.227-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
AsyncListener 2017-04-25T13:32:07.227-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:32:07.228-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-467729019
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:07.228-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:07.228-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:32:07.228-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:32:07.228-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:32:07.228-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:32:07.228-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:32:07.228-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:32:07.228-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:32:07.228-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:07.228573101 -0500 CDT
DcpNozzle 2017-04-25T13:32:07.228-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:07.228-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:07.228-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:32:07.229-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:07.228852746 -0500 CDT
AsyncListener 2017-04-25T13:32:07.229-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:07.229-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:07.229-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:32:07.229-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:32:07.229-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:32:07.230-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:32:07.230-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:32:07.230-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:32:07.230-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:32:07.230-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:07.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:32:07.230-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:07.230-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:32:07.230-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:07.230-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:32:07.230-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:32:07.230-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:07.230926192 -0500 CDT
DcpNozzle 2017-04-25T13:32:07.230-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:32:07.231-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:07.231-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:32:07.232-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:32:07.232-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:07.232858419 -0500 CDT
DcpNozzle 2017-04-25T13:32:07.232-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:32:07.232-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:07.232-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:32:07 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:32:07 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:32:07 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:32:07 Stream Closed for vb 1 StreamEnd simulated
XDCRFactory 2017-04-25T13:32:07.310-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:32:07.310-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:32:07.310-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:32:07.437-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:32:07.437-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:32:07.437-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:32:07.437-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:32:07.438-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:32:07.438-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:32:07.438-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:32:07.438-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:32:07.438-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
GenericSupervisor 2017-04-25T13:32:07.439-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:32:07.439-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
GenericSupervisor 2017-04-25T13:32:07.439-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:07.439-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:07.439-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:07.439-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:07.439-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:32:07.439-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
CheckpointManager 2017-04-25T13:32:07.439-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
CapiUtils 2017-04-25T13:32:07.488-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:32:07.488-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:32:07.488-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:32:07.489-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
StatisticsManager 2017-04-25T13:32:07.489-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:32:07.490-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
XDCRFactory 2017-04-25T13:32:07.490-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:32:07.490-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098 is constructed, start it
GenericPipeline 2017-04-25T13:32:07.491-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[257 233 240 185 209 225 272 289 189 190 213 295 250 264 227 258 262 284 211 237 321 337 217 210 218 300 191 301 182 310 336 286 175 206 243 249 329 171 197 339 174 266 176 204 319 322 334 292 298 216 239 313 178 279 326 224 269 324 200 304 246 256 265 275 173 283 327 183 315 331 223 293 177 195 261 316 202 215 231 312 172 255 268 186 188 234 287 288 194 314 323 274 193 235 248 340 184 205 192 201 271 263 198 276 306 335 181 282 338 230 238 296 196 207 242 277 320 252 303 179 221 229 232 273 307 302 222 241 247 285 180 203 267 270 308 290 281 187 208 214 251 278 333 199 226 245 317 325 259 220 254 291 294 311 228 260 309 318 330 332 280 212 236 219 297 299 244 253 305 328] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[115 89 122 125 159 41 30 24 40 86 141 4 58 143 149 36 12 49 50 72 79 137 7 113 129 146 5 121 53 164 44 48 87 78 130 166 69 81 43 84 117 168 42 118 14 93 136 162 11 139 67 70 31 65 83 101 111 144 148 160 46 59 147 150 6 124 151 56 80 105 64 39 107 19 76 119 135 20 98 91 35 62 95 163 18 16 47 63 71 153 15 21 165 8 75 26 99 114 156 169 10 60 128 34 103 32 123 74 132 157 57 167 116 97 120 82 73 161 27 145 96 110 127 66 85 108 77 45 142 9 140 52 22 54 152 2 29 38 55 61 68 92 158 23 154 33 104 106 134 155 37 17 25 100 1 112 51 94 0 126 131 170 102 88 109 13 28 90 133 138 3]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[196 230 238 296 242 277 207 320 252 303 221 229 232 273 179 302 307 247 285 222 241 180 203 267 270 290 308 281 214 251 278 333 187 208 199 226 245 259 317 325 220 254 291 294 311 309 318 330 332 228 260 212 280 236 219 299 297 244 253 305 328 257 233 240 225 272 289 185 209 295 189 190 213 264 250 227 258 262 284 211 217 237 321 337 191 210 218 300 301 336 182 310 286 175 206 243 249 329 339 171 197 174 266 322 334 176 204 319 292 298 313 216 239 178 279 326 269 324 224 304 200 256 265 275 246 173 183 283 327 315 331 177 223 293 195 261 316 202 172 215 231 312 255 268 186 188 194 234 287 288 323 314 274 340 184 193 235 248 205 192 201 271 263 306 335 198 276 338 181 282] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[27 73 161 145 66 96 110 127 77 85 108 9 45 142 52 140 2 22 54 152 61 68 92 158 23 29 38 55 33 154 155 37 104 106 134 1 17 25 100 51 112 0 94 102 126 131 170 13 88 109 138 3 28 90 133 115 159 41 89 122 125 24 30 40 4 86 141 36 58 143 149 72 79 137 7 12 49 50 5 113 129 146 53 121 44 164 48 87 69 78 130 166 43 81 42 84 117 168 118 162 11 14 93 136 67 139 31 70 111 144 148 160 46 65 83 101 6 59 147 150 56 124 151 64 80 105 19 39 107 76 20 119 135 91 98 163 18 35 62 95 71 153 15 16 47 63 8 21 165 75 156 169 10 26 99 114 34 60 128 32 103 74 123 132 57 157 167 97 116 82 120]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: target_nozzle_per_node:2 log_level:Info stats_interval:1000 source_nozzle_per_node:2 active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:32:07.491-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098...
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:32:07.531-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:32:07.531-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:07.531-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:32:07.531-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:32:07.531-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:32:07.531-05:00 [INFO] Checkpointing starts in 1802.544 sec
PipelineRuntimeCtx 2017-04-25T13:32:07.531-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:32:07.531-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:32:07.532-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:32:07.532-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:32:07.533-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:32:07.533-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:32:07.533-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:32:07.533-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:32:07.536-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:32:07.550-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:32:07.550-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:32:07.550-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:32:07.570-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:32:07.570-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:32:07.571-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:32:07.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:32:07.576-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:32:07.576-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:32:07.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:32:07.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:32:07.577-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:32:07.592-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:32:07.592-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:32:07.592-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CapiUtils 2017-04-25T13:32:07.594-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:32:07.594-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:32:07.594-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:32:07.603-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:32:07.603-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
ToplogyChangeDetector 2017-04-25T13:32:07.606-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:32:07.606-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:32:07.752-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:32:07.763-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:32:07.763-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:32:07.763-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:32:07.783-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:32:07.783-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:32:07.783-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:32:07.783-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098 is started----------
PipelineManager 2017-04-25T13:32:07.783-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:32:07.783-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:32:07.783-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:32:07.918-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:32:07.919-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:32:07.919-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:32:07.919-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:32:08 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:32:08.026-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:32:08.029-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:32:08.029-05:00 [INFO] Send batch count=56
CapiNozzle 2017-04-25T13:32:08.031-05:00 [INFO] Send batch count=118
CapiNozzle 2017-04-25T13:32:08.057-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:08.057-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:08.076-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:08.077-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:08.098-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:32:08.106-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:32:08 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 112 successful
CapiNozzle 2017-04-25T13:32:08.315-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:32:08 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 21 successful
CapiNozzle 2017-04-25T13:32:08.339-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:32:08 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 233 successful
CapiNozzle 2017-04-25T13:32:08.476-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:32:08 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 142 successful
CapiNozzle 2017-04-25T13:32:08.497-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:32:08.502-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:32:08 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 242 successful
CapiNozzle 2017-04-25T13:32:08.539-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:32:08 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 48 successful
StatisticsManager 2017-04-25T13:32:08.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=5, changes_left=315503
2017/04/25 13:32:08 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:32:08.839-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:32:08.857-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:32:08.876-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:32:08.876-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:32:08.876-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:32:08.876-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:32:08.876-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:32:08.890-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:32:08.890-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:32:08.896-05:00 [INFO] err_list=[{"time":"2017-04-25T13:32:08.896045205-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:32:08.896-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:32:08 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:32:08 UPR_STREAMREQ for vb 179 successful
CapiNozzle 2017-04-25T13:32:09.068-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:09.068-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:09.083-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:32:09 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 86 successful
CapiNozzle 2017-04-25T13:32:09.127-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:32:09.127-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:32:09.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:32:09.127-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:09.127-05:00 [INFO] Send batch count=500
2017/04/25 13:32:09 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 287 successful
CapiNozzle 2017-04-25T13:32:09.217-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:09.217-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:09.265-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:32:09 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 98 successful
CheckpointManager 2017-04-25T13:32:09.392-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
CheckpointManager 2017-04-25T13:32:09.392-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:32:09 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:32:09 UPR_STREAMREQ for vb 298 successful
DcpNozzle 2017-04-25T13:32:09.483-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:32:09.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:32:09.522-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
StatisticsManager 2017-04-25T13:32:09.536-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:32:09.537-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098 {"Errors": "[{\"time\":\"2017-04-25T13:32:08.896045205-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315503, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 77, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 5, "docs_received_from_dcp": 17906, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 77, "set_filtered": 0, "set_received_from_dcp": 17906, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:32:09.537-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:32:09.709-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:32:09.721-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:32:09.963-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:32:09.983-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:32:10.062-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:32:10.078-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:32:10.322-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:32:10.322-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:32:10.322-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:32:10.322-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:10.322-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:10.447-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:10.453-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:32:10.453-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:32:10.453-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:32:10.453-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:10.453-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:10.459-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:10.471-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:32:10.538-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:10.539-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:10.619-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:32:10.887-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:32:10.907-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:32:11.109-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:32:11.120-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:32:11.286-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:32:11.299-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:32:11.497-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:32:11.521-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:32:11.687-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:32:11.687-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:32:11.687-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:32:11.687-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:11.687-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:11.737-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:11.737-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:11.748-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:32:11.883-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:32:11.883-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:32:11.883-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:32:11.883-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:11.883-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:11.955-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:11.955-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:11.970-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:32:12.095-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:32:12.112-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:32:12.391-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:32:12.408-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:32:12.698-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:32:12.719-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:32:12.834-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:32:12.864-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:32:13.091-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:32:13.091-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:32:13.091-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:32:13.091-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:13.091-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:13.214-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:32:13.218-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:32:13.218-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:32:13.218-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:13.218-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:13.239-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:13.250-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:13.297-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:13.297-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:13.298-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:32:13.321-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:32:13.804-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:13.821-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:32:14.061-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:32:14.084-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:32:14.225-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:14.258-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:32:14.499-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:32:14.534-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:32:14.777-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:14.777-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:32:14.777-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:32:14.777-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:14.777-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:14.864-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:14.864-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:14.876-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:32:14.940-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:32:14.940-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:32:14.940-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:32:14.940-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:14.940-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:14.998-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:14.998-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:15.013-05:00 [INFO] Retrying update docs for vb 205 for the 1th time
CapiNozzle 2017-04-25T13:32:15.199-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:32:15.211-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:32:15.339-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:32:15.353-05:00 [INFO] Retrying update docs for vb 205 for the 2th time
CapiNozzle 2017-04-25T13:32:15.521-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:32:15.534-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:32:15.723-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:32:15.734-05:00 [INFO] Retrying update docs for vb 205 for the 3th time
PipelineManager 2017-04-25T13:32:15.844-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:32:08.896045205-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:32:16.011-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:32:16.011-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:32:16.011-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:32:16.011-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:16.011-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:16.120-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:16.120-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:16.127-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:32:16.127-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 205 after 3 retries
CapiNozzle 2017-04-25T13:32:16.127-05:00 [ERROR] Raise error condition batch update docs failed for vb 205 after 3 retries
GenericSupervisor 2017-04-25T13:32:16.127-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:16.127-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:16.131-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:32:16.226-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:16.227-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:16.238-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:32:16.592-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:32:16.602-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:32:16.730-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:32:16.740-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:32:16.957-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:32:16.968-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:32:17.245-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:32:17.256-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
GenericSupervisor 2017-04-25T13:32:17.532-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:32:17.532-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:32:17.532-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:32:17.586-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:32:17.586-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:32:17.586-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:32:17.586-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:17.586-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:17.709-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:17.709-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:17.720-05:00 [INFO] Retrying update docs for vb 107 for the 1th time
ToplogyChangeDetector 2017-04-25T13:32:17.853-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:32:17.861-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:32:17.861-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:32:17.861-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:32:17.862-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:17.862-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:17.995-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:17.995-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:18.006-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
CapiNozzle 2017-04-25T13:32:18.085-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:32:18.097-05:00 [INFO] Retrying update docs for vb 107 for the 2th time
CapiNozzle 2017-04-25T13:32:18.444-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:32:18.455-05:00 [INFO] Retrying update docs for vb 208 for the 2th time
CapiNozzle 2017-04-25T13:32:18.613-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:32:18.624-05:00 [INFO] Retrying update docs for vb 107 for the 3th time
CapiNozzle 2017-04-25T13:32:18.833-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:32:18.845-05:00 [INFO] Retrying update docs for vb 208 for the 3th time
PipelineManager 2017-04-25T13:32:18.896-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:32:18.896-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:32:18.896-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:32:18.896-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
PipelineRuntimeCtx 2017-04-25T13:32:18.896-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:32:18.896-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:18.896-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:32:18.896-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:18.896634089 -0500 CDT
GenericSupervisor 2017-04-25T13:32:18.896-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:18.896-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:32:18.896-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:32:18.896-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:32:18.896-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:32:18.896-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:32:18.896-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:32:18.896-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:32:18.896-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:32:18.896-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-438073098 has been stopped
GenericPipeline 2017-04-25T13:32:18.897-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-438073098 is stopped
PipelineManager 2017-04-25T13:32:18.897-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:32:18.897-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:32:08.896045205-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:32:18.898-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:32:18.899-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
GenericPipeline 2017-04-25T13:32:18.899-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
GenericPipeline 2017-04-25T13:32:18.899-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
GenericPipeline 2017-04-25T13:32:18.899-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-438073098
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:18.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:32:18.899-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:18.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:32:18.899-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:32:18.899-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:32:18.900-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:32:18.900-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:18.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:32:18.900-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:32:18.900-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:32:18.900-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:18.900-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:18.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:32:18.900-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:18.900122666 -0500 CDT
DcpNozzle 2017-04-25T13:32:18.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:32:18.900-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:18.900510744 -0500 CDT
CapiNozzle 2017-04-25T13:32:18.901-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:32:18.901-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:32:18.901-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:32:18.901-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:18.901-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:18.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:32:18.902-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:18.902-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:32:18.905-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:32:18.905-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:32:18.905-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:18.905937745 -0500 CDT
DcpNozzle 2017-04-25T13:32:18.906-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:18.905955747 -0500 CDT
DcpNozzle 2017-04-25T13:32:18.906-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:32:18.906-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:32:18.906-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:18.906-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:32:18.906-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:18.906-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:32:18 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:32:18 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:32:18 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:32:18.962-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:32:18.963-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:32:18.963-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:32:18.963-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:32:19.070-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:32:19.071-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:32:19.071-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:32:19.125-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:32:19.125-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:32:19.125-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:32:19.125-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:32:19.126-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:32:19.126-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:32:19.126-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:32:19.126-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:32:19.126-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
GenericSupervisor 2017-04-25T13:32:19.128-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:32:19.128-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
GenericSupervisor 2017-04-25T13:32:19.128-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:19.128-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:19.128-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:19.128-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:19.128-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:32:19.128-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
CheckpointManager 2017-04-25T13:32:19.128-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
CapiUtils 2017-04-25T13:32:19.168-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:32:19.169-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:32:19.169-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:32:19.171-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
StatisticsManager 2017-04-25T13:32:19.171-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:32:19.171-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
XDCRFactory 2017-04-25T13:32:19.171-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:32:19.171-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627 is constructed, start it
GenericPipeline 2017-04-25T13:32:19.173-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[95 118 10 51 52 68 168 8 79 80 144 147 130 2 39 45 78 90 62 73 55 128 44 67 103 137 138 7 12 13 50 71 74 96 101 23 115 169 25 61 20 100 129 9 29 40 76 64 156 1 116 4 11 63 160 35 134 16 155 165 166 107 154 150 42 131 158 84 94 17 104 135 125 133 162 19 98 108 110 113 170 75 112 58 153 14 102 145 146 65 70 48 60 119 30 57 59 99 126 24 33 83 92 148 36 121 139 143 77 91 152 6 72 87 161 46 114 149 151 136 141 157 82 85 167 88 106 111 132 32 164 3 66 123 142 28 49 54 81 127 0 38 109 41 105 56 117 5 120 18 21 86 159 89 27 53 15 93 22 31 34 47 37 69 122 163 26 43 97 124 140] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[244 259 243 293 313 206 192 213 232 198 277 291 220 227 294 185 199 285 175 253 263 189 191 197 208 295 319 334 207 225 266 286 204 317 240 297 298 264 276 209 226 325 278 304 179 217 247 281 270 251 303 324 184 188 284 318 321 228 257 267 309 214 288 314 248 323 181 250 265 331 336 216 246 301 306 332 212 224 233 312 302 221 255 219 237 256 271 329 339 327 337 186 245 340 322 316 320 171 178 272 287 315 238 308 177 190 202 239 292 330 215 172 261 326 180 211 254 218 279 311 176 269 333 182 235 241 283 300 234 328 173 229 305 183 193 289 249 307 174 187 230 280 296 299 201 273 282 194 203 242 274 310 223 231 268 258 260 338 200 205 236 252 196 275 262 195 210 222 290 335]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[186 245 340 322 316 320 171 178 272 287 315 238 308 177 190 202 239 292 330 215 172 261 326 180 211 254 218 279 311 176 269 333 182 235 241 283 300 234 328 173 229 305 183 193 289 249 307 174 187 230 280 296 299 201 273 282 194 203 242 274 310 223 231 268 258 260 338 200 205 236 252 196 275 262 195 210 222 290 335 244 259 243 293 313 206 192 213 232 198 277 291 220 227 294 185 199 285 175 253 263 189 191 197 208 295 319 334 207 225 266 286 204 317 240 297 298 264 276 209 226 325 278 304 179 217 247 281 270 251 303 324 184 188 284 318 321 228 257 267 309 214 288 314 248 323 181 250 265 331 336 216 246 301 306 332 212 224 233 312 302 221 255 219 237 256 271 329 339 327 337] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[36 121 139 143 77 91 152 6 72 87 161 46 114 149 151 136 141 157 82 85 167 88 106 111 132 32 164 3 66 123 142 28 49 54 81 127 0 38 109 41 105 56 117 5 120 18 21 86 159 89 27 53 15 93 22 31 34 47 37 69 122 163 26 43 97 124 140 95 118 10 51 52 68 168 8 79 80 144 147 130 2 39 45 78 90 62 73 55 128 44 67 103 137 138 7 12 13 50 71 74 96 101 23 115 169 25 61 20 100 129 9 29 40 76 64 156 1 116 4 11 63 160 35 134 16 155 165 166 107 154 150 42 131 158 84 94 17 104 135 125 133 162 19 98 108 110 113 170 75 112 58 153 14 102 145 146 65 70 48 60 119 30 57 59 99 126 24 33 83 92 148]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 optimistic_replication_threshold:256 log_level:Info target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:32:19.173-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627...
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:32:19.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:32:19.191-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:19.191-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:32:19.191-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:32:19.191-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:32:19.191-05:00 [INFO] Checkpointing starts in 1804.83 sec
PipelineRuntimeCtx 2017-04-25T13:32:19.191-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:32:19.191-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:32:19.192-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:32:19.192-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:32:19.195-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:32:19.195-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:32:19.195-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:32:19.195-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:32:19.199-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:32:19.248-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:32:19.256-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:32:19.256-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:32:19.256-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:32:19.289-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:32:19.289-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:32:19.291-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:32:19.291-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:32:19.291-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:32:19.319-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:32:19.320-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:32:19.403-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:32:19.404-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:32:19.404-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:32:19.404-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:32:19.404-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:32:19.404-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:32:19.429-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:32:19.429-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:32:19.429-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:32:19.501-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:32:19.520-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:32:19.520-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:32:19.693-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:32:19.693-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:32:19.693-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:32:19.732-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:32:19.732-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:32:19.732-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:32:19.732-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627 is started----------
PipelineManager 2017-04-25T13:32:19.732-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:32:19.732-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:32:19.732-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:32:19.956-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:32:19.956-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:32:19.957-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:32:19.957-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:32:20 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 112 successful
StatisticsManager 2017-04-25T13:32:20.260-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:32:20 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 322 successful
CapiNozzle 2017-04-25T13:32:20.478-05:00 [INFO] Send batch count=498
2017/04/25 13:32:20 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 226 successful
CapiNozzle 2017-04-25T13:32:20.572-05:00 [INFO] Send batch count=175
CapiNozzle 2017-04-25T13:32:20.586-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:20.586-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:20.606-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:32:20 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:32:20 UPR_STREAMREQ for vb 275 successful
CapiNozzle 2017-04-25T13:32:20.995-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:32:21.029-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
2017/04/25 13:32:21 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 194 successful
CheckpointManager 2017-04-25T13:32:21.175-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
CheckpointManager 2017-04-25T13:32:21.175-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:32:21.200-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:32:21 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 96 successful
CapiNozzle 2017-04-25T13:32:21.211-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
StatisticsManager 2017-04-25T13:32:21.277-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:32:21 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:32:21 UPR_STREAMREQ for vb 298 successful
CapiNozzle 2017-04-25T13:32:21.290-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
DcpNozzle 2017-04-25T13:32:21.310-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:32:21.310-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
DcpNozzle 2017-04-25T13:32:21.336-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:32:21.832-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:32:21.850-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:32:21.950-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:32:21.986-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
StatisticsManager 2017-04-25T13:32:22.255-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:32:22.263-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:32:22.263-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:32:22.263-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:32:22.263-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:32:22.263-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:32:22.360-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:32:22.360-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:32:22.363-05:00 [INFO] err_list=[{"time":"2017-04-25T13:32:22.363587781-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:32:22.363-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:32:22.423-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:32:22.423-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:32:22.423-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:32:22.423-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:22.423-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:22.954-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:22.955-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:22.963-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:22.963-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:22.974-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:32:22.984-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
StatisticsManager 2017-04-25T13:32:23.200-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:32:23.202-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627 {"Errors": "[{\"time\":\"2017-04-25T13:32:22.363587781-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 360, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 27062, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 360, "set_filtered": 0, "set_received_from_dcp": 27062, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:32:23.269-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:32:23.280-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:32:23.432-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:23.445-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:32:23.651-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:32:23.678-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:32:23.836-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:23.847-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:32:23.928-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:32:23.975-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:32:24.098-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:32:24.166-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:27.861-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:32:29.488-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:32:29.488-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:32:29.488-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:32:29.488-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:32:29.488-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:32:29.497-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:32:29.497-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:32:29.504-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:32:29.641-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:32:29.690-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:32:29.694-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:32:30.345-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:32:30.634-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
PipelineManager 2017-04-25T13:32:32.539-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:32:32.824-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:32:32.690-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:32:32.832-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:32:32.832-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
PipelineRuntimeCtx 2017-04-25T13:32:32.832-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:32:32.832-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:32.833-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:32:32.833-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:32.832579037 -0500 CDT
GenericSupervisor 2017-04-25T13:32:32.852-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:32.852-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:32:32.853-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:32:32.853-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:32:32.853-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:32:32.853-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:32:32.853-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:32:32.853-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:32:32.853-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:32:32.853-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-126137627 has been stopped
GenericPipeline 2017-04-25T13:32:32.853-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-126137627 is stopped
PipelineManager 2017-04-25T13:32:32.853-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:32:32.853-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:32:22.363587781-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:32:32.854-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:32:32.854-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
GenericPipeline 2017-04-25T13:32:32.854-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
GenericPipeline 2017-04-25T13:32:32.854-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
GenericPipeline 2017-04-25T13:32:32.854-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-126137627
AsyncListener 2017-04-25T13:32:32.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:32.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:32.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:32.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:32.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:32.854-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:32.855-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:32.855-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:32:32.855-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:32.855-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:32.858-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:32:32.858-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:32:32.859-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:32.859-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:32:32.859-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:32:32.862-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:32:32.862-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:32:32.863-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:32:32.865-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:32.866-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:32:32.866-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:32:32.867-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:32:32.867-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:32:32.868-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.875-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:32:32.876-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:32:32.876-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:32:32.882-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:32.865985034 -0500 CDT
DcpNozzle 2017-04-25T13:32:32.888-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:32:32.888-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:32.875429699 -0500 CDT
CapiNozzle 2017-04-25T13:32:32.888-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:32:32.889-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:32:32.889-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:32.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:32.902-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:32:32.903-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:32:32.904-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
DcpNozzle 2017-04-25T13:32:33.390-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:32:33 Stream Closed for vb 171 StreamEnd simulated
GenericPipeline 2017-04-25T13:32:34.356-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:32:34.529-05:00 [INFO] ****************************
DcpNozzle 2017-04-25T13:32:33.232-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:32:34.311-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:32:34.205-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:32:34.328-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:32:34.747-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:32:34.741-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:32:33.842-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:32:34.806-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:32:35.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
GenericPipeline 2017-04-25T13:32:34.960-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:32:35.306-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:32:35.006-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
DcpNozzle 2017-04-25T13:32:35.131-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:34.826590885 -0500 CDT
GenericPipeline 2017-04-25T13:32:35.354-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:32:34.980-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:32:35.481-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:32:34.729-05:00 [INFO] ****************************
DcpNozzle 2017-04-25T13:32:34.984-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:32:34.850718342 -0500 CDT
GenericPipeline 2017-04-25T13:32:35.537-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:32:35.537-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
DcpNozzle 2017-04-25T13:32:35.459-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:32:35.560-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:32:35.603-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:32:35.615-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:32:36.661-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:32:36.714-05:00 [INFO] releasing capi client
GenericSupervisor 2017-04-25T13:32:36.858-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:32:36.962-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
PipelineManager 2017-04-25T13:32:37.003-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:32:22.363587781-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:32:37.837-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:32:37.840-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:32:37.840-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:32:38.234-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:32:39.999-05:00 [INFO] Constructed 2 outgoing nozzles
ReplicationManager 2017-04-25T13:32:39.358-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
XDCRRouter 2017-04-25T13:32:41.994-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:32:42.002-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:32:42.003-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:32:42.003-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:32:42.003-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:32:42.003-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:32:42.004-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
GenericSupervisor 2017-04-25T13:32:42.007-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:32:42.008-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
GenericSupervisor 2017-04-25T13:32:42.008-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:42.008-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:42.008-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:42.008-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:32:42.008-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:32:42.008-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
CheckpointManager 2017-04-25T13:32:42.008-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
CapiUtils 2017-04-25T13:32:43.010-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:32:43.010-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:32:43.010-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:32:43.057-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
StatisticsManager 2017-04-25T13:32:43.150-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:32:43.189-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
XDCRFactory 2017-04-25T13:32:43.189-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:32:43.240-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 is constructed, start it
GenericPipeline 2017-04-25T13:32:43.245-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[33 82 160 163 29 125 0 148 86 103 34 122 47 118 133 112 42 68 1 114 150 59 94 126 140 58 88 102 113 158 13 44 138 168 26 141 12 93 8 84 66 17 77 169 57 165 54 155 170 25 89 142 147 71 56 83 41 64 157 39 116 74 151 55 80 106 130 137 139 162 75 144 146 70 31 98 123 135 30 11 134 7 159 9 22 48 132 20 51 99 107 128 50 104 96 24 49 149 6 105 156 121 28 37 101 115 10 85 90 2 73 81 109 131 53 127 167 67 92 27 61 108 110 4 143 5 78 153 166 45 152 154 16 40 95 124 164 21 35 63 18 69 72 161 43 38 15 62 120 3 46 111 136 36 87 65 117 23 145 97 91 129 52 76 79 119 32 19 60 100 14] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[189 238 213 245 291 219 276 277 284 319 186 192 281 295 211 249 289 177 218 270 317 331 313 241 323 215 237 255 266 314 340 178 171 184 247 272 328 334 252 206 282 318 250 330 196 176 201 305 316 233 329 288 335 216 207 275 253 286 222 224 306 197 223 246 336 183 191 292 297 193 198 290 194 226 301 210 312 174 285 299 203 308 179 185 202 300 180 230 262 256 264 311 325 240 309 320 229 231 271 283 307 236 212 232 280 214 258 265 279 234 267 199 204 259 298 326 190 235 296 327 338 217 225 248 268 260 273 227 172 263 339 239 243 304 321 322 181 173 188 195 205 257 293 337 303 175 182 209 221 310 200 274 278 228 242 254 294 302 324 332 187 244 261 220 269 287 333 208 251 315]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[86 148 34 103 122 47 112 118 133 1 42 68 59 114 150 58 94 126 140 13 88 102 113 158 26 44 138 168 12 141 8 93 66 84 17 57 77 169 165 25 54 155 170 71 89 142 147 41 56 83 39 64 157 74 116 55 151 80 75 106 130 137 139 162 70 144 146 30 31 98 123 135 7 11 134 9 159 20 22 48 132 50 51 99 107 128 96 104 6 24 49 149 105 121 156 10 28 37 101 115 2 85 90 53 73 81 109 131 67 127 167 92 4 27 61 108 110 5 143 45 78 153 166 16 152 154 21 40 95 124 164 18 35 63 69 43 72 161 38 3 15 62 120 36 46 111 136 87 23 65 117 145 97 52 91 129 32 76 79 119 14 19 60 100 29 33 82 160 163 0 125] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[186 219 276 277 284 319 192 281 295 211 249 289 177 218 270 317 331 313 241 323 215 237 255 266 314 340 178 272 171 184 247 328 334 252 206 282 318 250 330 196 176 201 305 316 233 329 288 335 216 207 275 253 286 222 224 306 197 223 246 336 183 191 292 297 198 193 290 194 226 301 210 312 174 285 299 203 308 202 179 185 300 180 230 262 325 256 264 311 240 309 320 229 231 271 283 307 236 212 232 280 214 258 265 279 234 267 259 199 204 298 190 326 338 235 296 327 217 225 248 268 260 273 227 172 263 339 181 239 243 304 321 322 173 188 293 195 205 257 337 303 175 182 209 221 310 200 274 278 332 228 242 254 294 302 324 187 244 261 220 269 287 333 208 251 315 189 238 213 245 291]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 filter_expression: checkpoint_interval:1800 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi active:true]
CheckpointManager 2017-04-25T13:32:43.251-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084...
AsyncListener 2017-04-25T13:32:43.319-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:32:43.320-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:32:43.492-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:32:43.558-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:32:43.558-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:32:43.761-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:32:43.761-05:00 [INFO] Checkpointing starts in 1801.733 sec
GenericSupervisor 2017-04-25T13:32:43.630-05:00 [INFO] monitorPipelineHealth started
PipelineRuntimeCtx 2017-04-25T13:32:43.806-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:32:43.822-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:32:43.910-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:32:43.910-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:32:43.910-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:32:43.907-05:00 [INFO] checkpointing rountine started
CheckpointManager 2017-04-25T13:32:43.948-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:32:44.029-05:00 [INFO] Service StatisticsManager has been started
CheckpointManager 2017-04-25T13:32:44.037-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:32:44.038-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:32:44.065-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:32:44.065-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:32:44.100-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
StatisticsManager 2017-04-25T13:32:44.151-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:32:44.386-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:32:52.377-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CheckpointManager 2017-04-25T13:32:53.235-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:32:53.239-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CheckpointManager 2017-04-25T13:32:53.189-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CapiNozzle 2017-04-25T13:32:53.307-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CheckpointManager 2017-04-25T13:32:53.209-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:32:53.126-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
DcpNozzle 2017-04-25T13:32:53.467-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:32:53.471-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
GenericSupervisor 2017-04-25T13:32:54.469-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:32:55.797-05:00 [INFO] Executing 0x681fb0 timed out
StatisticsManager 2017-04-25T13:32:56.326-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:32:56.325-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:32:56.633-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:32:57.003-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:32:22.363587781-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:32:57.127-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:32:57.127-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:32:57.222-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:32:57.227-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:32:57.227-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:32:57.227-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:32:57.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
DcpNozzle 2017-04-25T13:32:57.639-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:32:57.928-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:32:57.938-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericSupervisor 2017-04-25T13:32:57.888-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:32:58.117-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
DcpNozzle 2017-04-25T13:32:57.867-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:32:58.290-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
GenericSupervisor 2017-04-25T13:32:58.226-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:32:58.366-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:32:58 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:32:58.619-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
GenericSupervisor 2017-04-25T13:32:58.843-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:32:59.777-05:00 [INFO] Send batch count=1
GenericSupervisor 2017-04-25T13:32:59.835-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:33:04.030-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:33:00.152-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:33:04.196-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
GenericSupervisor 2017-04-25T13:33:04.090-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:33:04.333-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:33:04.339-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
GenericSupervisor 2017-04-25T13:33:05.055-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:33:04.955-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
GenericSupervisor 2017-04-25T13:33:05.128-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:33:06.288-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:33:06.258-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:33:06.114-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
GenericSupervisor 2017-04-25T13:33:06.468-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:33:08.210-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:33:10.762-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:33:10.275-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:33:11.359-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:33:11.203-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:33:11.958-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:33:12.122-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:32:22.363587781-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 290, "dcp_dispatch_time": 1928, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 2902, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 640, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 656998, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 270, "dcp_dispatch_time": {"count": 621, "max": 798313, "mean": 1928.2415458937198, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:33:12.227-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 823 items, sent 501 items
StatisticsManager 2017-04-25T13:33:12.910-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:33:13.047-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:912]
StatisticsManager 2017-04-25T13:33:14.028-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 912 items, sent 912 items. streams inactive: [90 132 157 23 32 41 43 50 109 130 36 52 78 82 101 13 8 19 76 108 113 10 34 37 83 163 12 73 107 114 141 137 138 153 35 39 42 75 123 155 167 169 148 77 97 104 117 125 7 9 149 89 116 118 126 147 142 5 11 14 26 131 57 66 100 135 38 44 65 166 168 3 20 111 144 165 30 28 64 98 105 99 120 121 6 72 85 86 91 128 161 70 93 102 21 45 48 55 60 133 69 92 106 2 4 47 49 62 33 59 68 87 134 18 24 63 61 103 162 127 140 154 17 25 58 81 94 164 112 139 143 1 15 16 31 79 170 27 51 54 158 136 145 152 56 156 22 74 159 160 124 150 29 88 95 110 119 80 115 129 151 40 46 71 96 53 67 84 122 146]
StatisticsManager 2017-04-25T13:33:14.167-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:33:14.167-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
GenericSupervisor 2017-04-25T13:33:14.190-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =911 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =316 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:14.231-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
DcpNozzle 2017-04-25T13:33:14.464-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:33:15.122-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:33:15.281-05:00 [INFO] Dcp nozzle is started
GenericSupervisor 2017-04-25T13:33:15.809-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:33:16.062-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:33:16.068-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:33:16.042-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
GenericPipeline 2017-04-25T13:33:16.288-05:00 [INFO] All parts has been started
DcpNozzle 2017-04-25T13:33:15.146-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
2017/04/25 13:33:16 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:33:16 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:33:16 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:33:16 UPR_STREAMREQ for vb 100 successful
GenericSupervisor 2017-04-25T13:33:20.464-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:33:20.997-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:33:21.053-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:33:20.841-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:33:21.093-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:33:21.096-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:33:21.099-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 is started----------
PipelineManager 2017-04-25T13:33:21.105-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:33:21.105-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:33:21.126-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[{"time":"2017-04-25T13:32:22.363587781-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is running}
]
StatisticsManager 2017-04-25T13:33:21.207-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:32:22.363587781-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13941, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 411, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 695563, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 1711}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 411, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 695563, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 418, "max": 9735, "mean": 1711.6985645933014, "min": 1690}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 689, "dcp_dispatch_time": {"count": 912, "max": 3945207, "mean": 13941.732456140351, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 4, "expiry_received_from_dcp": 0, "set_received_from_dcp": 4}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 302, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 1, "expiry_received_from_dcp": 0, "set_received_from_dcp": 1}}

StatisticsManager 2017-04-25T13:33:21.209-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 1825 items, sent 501 items
StatisticsManager 2017-04-25T13:33:21.209-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 2102 items, sent 500 items
StatisticsManager 2017-04-25T13:33:21.209-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:1825]
PipelineManager 2017-04-25T13:33:21.210-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 1825 items, sent 1825 items. streams inactive: [10 34 37 83 141 163 12 73 107 114 123 137 138 153 35 39 42 75 155 167 169 125 148 77 97 104 117 7 9 149 147 89 116 118 126 131 142 5 11 14 26 57 66 135 38 44 65 166 165 168 3 20 111 144 30 28 64 98 105 91 99 120 121 6 72 85 86 128 161 60 70 93 102 21 45 48 55 133 62 69 92 106 2 4 47 49 134 33 59 68 87 18 24 63 61 103 162 94 127 140 154 17 25 58 81 164 79 112 139 143 1 15 16 31 170 27 51 54 158 136 145 152 56 156 160 22 74 159 119 124 150 29 88 95 110 80 115 129 151 40 46 71 96 146 53 67 84 122 50 90 132 157 23 32 41 43 101 109 130 36 52 78 82 13 113 8 19 76 108]
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:2105]
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 2106 items, sent 2106 items. streams inactive: [192 235 262 311 304 317 339 211 247 253 289 291 269 335 219 237 277 329 288 312 171 178 179 252 280 327 183 332 201 238 263 212 215 242 307 315 316 320 314 198 214 294 313 338 340 232 249 267 279 325 223 259 290 322 191 221 224 268 251 310 326 333 264 276 305 188 189 194 206 230 318 321 177 186 203 239 265 308 337 270 295 296 193 196 243 244 261 319 323 181 207 226 299 302 278 292 330 187 210 218 256 272 190 233 241 260 324 225 284 287 309 286 293 195 202 209 236 285 306 174 208 227 234 274 197 222 231 245 281 258 173 180 229 250 257 266 282 298 175 199 204 228 248 182 216 271 275 328 331 205 217 220 246 273 172 185 254 283 297 336 176 184 213 240 255 303 334]
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =316 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =11 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =2115 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =1821 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:33:21.212-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
2017/04/25 13:33:21 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:33:21.472-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:33:21 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:33:21.580-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:33:21 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 102 successful
StatisticsManager 2017-04-25T13:33:21.593-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
ToplogyChangeDetector 2017-04-25T13:33:21.609-05:00 [INFO] validateSourceTopology...
2017/04/25 13:33:21 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 5 successful
CapiNozzle 2017-04-25T13:33:21.778-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:33:21 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:33:21 UPR_STREAMREQ for vb 205 successful
CapiNozzle 2017-04-25T13:33:21.871-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
GenericSupervisor 2017-04-25T13:33:24.463-05:00 [INFO] About to call function
2017/04/25 13:33:24 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:33:24 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:33:24 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:33:24 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:33:24 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:33:24 UPR_STREAMREQ for vb 106 successful
DcpNozzle 2017-04-25T13:33:24.424-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
DcpNozzle 2017-04-25T13:33:24.650-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
CapiNozzle 2017-04-25T13:33:24.826-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericSupervisor 2017-04-25T13:33:36.853-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:33:37.197-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:33:41.805-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:33:41.346-05:00 [ERROR] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 - Connection reset failed
CapiNozzle 2017-04-25T13:33:42.325-05:00 [ERROR] Raise error condition dial tcp 10.202.98.33:9091: i/o timeout
GenericSupervisor 2017-04-25T13:33:40.326-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:33:43.398-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:33:43.465-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout]
ReplicationManager 2017-04-25T13:33:43.905-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout]
GenericSupervisor 2017-04-25T13:33:43.768-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:33:44.410-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:33:44.413-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:33:44.433-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:33:44.433-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:33:44 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:33:44 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:33:44 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:33:44 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:33:44 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:33:44 UPR_STREAMREQ for vb 308 successful
PipelineManager 2017-04-25T13:33:44.871-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
DcpNozzle 2017-04-25T13:33:44.875-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:33:44.875-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
2017/04/25 13:33:44 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:33:44 UPR_STREAMREQ for vb 208 successful
StatisticsManager 2017-04-25T13:33:44.900-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:33:44.901-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:33:44.902-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-3336084 {"Errors": "[{\"time\":\"2017-04-25T13:33:44.872921424-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 1648, "dcp_dispatch_time": 8649, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 630, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 12977, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 630, "set_filtered": 0, "set_received_from_dcp": 12977, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:33:44.912-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
PipelineManager 2017-04-25T13:33:44.872-05:00 [INFO] err_list=[{"time":"2017-04-25T13:33:44.872921424-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout"}]
PipelineManager 2017-04-25T13:33:44.932-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:33:44 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:33:45 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:33:45 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:33:45 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:33:45 UPR_STREAMREQ for vb 209 successful
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x6b324d]

goroutine 3327 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).tcpProxy(0xc393c26480, 0xc8, 0xc2bf57ae40, 0xc29f614c40, 0xc2bf57ad80, 0xc2bf57ade0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:908 +0x7cd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:831 +0xc91

goroutine 1 [chan receive, 3 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 3 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 3 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc2080d6370)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc20806e420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc20806e420, 0xc2080dc001, 0x5ff, 0x5ff, 0xc2080da080, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080dc001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080d2000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d2000, 0x8f07a0, 0xc20806e4a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc20806e480, 0xc2080d85e0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bc00, 0x7fb4f722ad08, 0xc20806e480, 0x0, 0x0, 0xc2080d85e0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bc00, 0x7fb4f722ad08, 0xc20806e480, 0xc208084280, 0xc20801f660, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bc00, 0x7fb4f722ad08, 0xc20806e480)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b5f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b5f90, 0xc2080be480, 0x7fb4f7229490, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 16 [syscall, 3 minutes]:
syscall.Syscall(0x0, 0x0, 0xc208121000, 0x1000, 0xc20803b270, 0xc208032460, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc208121000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc208121000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc208121000, 0x1000, 0x1000, 0xc20803b270, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc208121000, 0x1000, 0x1000, 0xc2080f5630, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc2081109c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc2081109c0, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 33 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc20810a2a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 11 [select, 3 minutes]:
net/http.(*persistConn).readLoop(0xc20807e6e0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 12 [select, 3 minutes]:
net/http.(*persistConn).writeLoop(0xc20807e6e0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3095 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc393af2480, 0xc39874c5d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 34 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc20810a2b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 35 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 36 [select, 3 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 37 [select, 3 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080f7c00, 0x11, 0xc208151fc0, 0xc2081105a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080f7c00, 0x11, 0xc208151fc0, 0xc2081105a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080f7c00, 0x11, 0xc208151fc0, 0xc2081105a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080f4ff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 38 [select, 3 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080d7cb0, 0xf, 0xc208150fc0, 0xc2081105a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080d7cb0, 0xf, 0xc208150fc0, 0xc2081105a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080d7cb0, 0xf, 0xc208150fc0, 0xc2081105a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080f5180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 39 [runnable]:
fmt.(*pp).argNumber(0xc3a62b2680, 0x1, 0xaa2ff0, 0x33, 0x14, 0x4, 0x0, 0x7, 0xc264c3b800)
	/usr/local/go/src/fmt/print.go:1046
fmt.(*pp).doPrintf(0xc3a62b2680, 0xaa2ff0, 0x33, 0xc29475cdd8, 0x4, 0x4)
	/usr/local/go/src/fmt/print.go:1102 +0xe91
fmt.Sprintf(0xaa2ff0, 0x33, 0xc29475cdd8, 0x4, 0x4, 0x0, 0x0)
	/usr/local/go/src/fmt/print.go:203 +0x78
github.com/couchbase/goxdcr/pipeline.(*ReplicationStatus).String(0xc2080eca20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/ReplicationStatus.go:307 +0x259
fmt.(*pp).handleMethods(0xc3a6176b60, 0x76, 0x1, 0x9a2601)
	/usr/local/go/src/fmt/print.go:720 +0x579
fmt.(*pp).printValue(0xc3a6176b60, 0x9a2640, 0xc2080eca20, 0x16, 0xc200000076, 0x1, 0x9a2640)
	/usr/local/go/src/fmt/print.go:832 +0x412
fmt.(*pp).printReflectValue(0xc3a6176b60, 0x835dc0, 0xc264c3b590, 0x15, 0x76, 0x0, 0x0)
	/usr/local/go/src/fmt/print.go:890 +0x2f49
fmt.(*pp).printArg(0xc3a6176b60, 0x835dc0, 0xc264c3b590, 0x76, 0x0, 0x0)
	/usr/local/go/src/fmt/print.go:798 +0x44f
fmt.(*pp).doPrintf(0xc3a6176b60, 0xc26c2d6b80, 0x3e, 0xc29475dd30, 0x1, 0x1)
	/usr/local/go/src/fmt/print.go:1183 +0x21cd
fmt.Sprintf(0xc26c2d6b80, 0x3e, 0xc29475dd30, 0x1, 0x1, 0x0, 0x0)
	/usr/local/go/src/fmt/print.go:203 +0x78
log.(*Logger).Printf(0xc2080f4c80, 0xc26c2d6b80, 0x3e, 0xc29475dd30, 0x1, 0x1)
	/usr/local/go/src/log/log.go:160 +0x50
github.com/couchbase/goxdcr/log.(*CommonLogger).logMsgf(0xc2080d7b90, 0x1, 0x9c66e0, 0x7, 0xa3e690, 0x18, 0xc29475dd30, 0x1, 0x1)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:137 +0xf6
github.com/couchbase/goxdcr/log.(*CommonLogger).Infof(0xc2080d7b90, 0xa3e690, 0x18, 0xc29475dd30, 0x1, 0x1)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:148 +0x76
github.com/couchbase/goxdcr/pipeline_manager.LogStatusSummary()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:167 +0xe1
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:229 +0x2c7
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc2080e2930)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 40 [select]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208110960)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:102 +0x75b
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 23 [select]:
net/http.(*persistConn).roundTrip(0xc39d45c2c0, 0xc2dcc17b70, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc26008f040, 0xc266b08b70, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc26008f040, 0x7fb4f7228fe8, 0xc20802c090, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc26008f040, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc26008f040, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc26008f040, 0xeb0d80, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/goxdcr/utils.doRestCall(0xc26008f040, 0x0, 0x818220, 0xc39d94e5f0, 0xeb0d80, 0xc20802b450, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc268ecd5e0, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:173 +0x201
github.com/couchbase/goxdcr/utils.QueryRestApi(0xc268ecd5e0, 0x15, 0x9e0430, 0xc, 0xc3a03cb600, 0x9bf4c0, 0x3, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:134 +0x17a
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostInfo(0xc20801f440, 0xc266b08660, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:113 +0x139
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:157 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostName(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:177 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyHost(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:50 +0x4c
github.com/couchbase/goxdcr/replication_manager.GetReplicationInfos(0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:507 +0x304
github.com/couchbase/goxdcr/replication_manager.(*Adminport).doGetAllReplicationInfosRequest(0xc208110960, 0xc26377cea0, 0xc266b082a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:361 +0x73
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc208110960, 0xc26377cea0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:188 +0x7d9
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc208110960, 0xc20c62fd40, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc20c62fd40, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208110978)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 21 [select, 3 minutes]:
net/http.(*persistConn).readLoop(0xc20810c000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 22 [select, 3 minutes]:
net/http.(*persistConn).writeLoop(0xc20810c000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 24 [IO wait]:
net.(*pollDesc).Wait(0xc208010920, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010920, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080108c0, 0x0, 0x7fb4f7228df0, 0xc2ddfce760)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038128, 0xc20805fe38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208038128, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc20806ed80, 0x7fb4f722c7c0, 0xc208038128, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 25 [IO wait, 3 minutes]:
net.(*pollDesc).Wait(0xc2080111e0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080111e0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011180, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc20802b758)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038140, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc208038140, 0xc20807e738, 0xc2080c5000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801f740, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806e7e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc20806e7e0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc20806e7e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf200)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf200, 0xc208160000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080daf00, 0xc208160000, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080daf00, 0xc208160000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080daf40, 0xc208160000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d2480, 0xc2080da7c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d2480, 0x80f160, 0xc2080db000, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 26 [IO wait, 3 minutes]:
net.(*pollDesc).Wait(0xc208010760, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010760, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010700, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc20802b890)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380d0, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc2080380d0, 0xc20810c058, 0xc2080c3000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080ea240, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806eae0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc20806eae0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc20806eae0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf2f0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf2f0, 0xc208160600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080db080, 0xc208160600, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080db080, 0xc208160600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080db0c0, 0xc208160600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d2600, 0xc2080da980, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d2600, 0x80f160, 0xc2080db180, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 398 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x0, 0xc21826da00, 0xea, 0x100, 0xc21740e800, 0xb8, 0x100, 0xc2184e41e0, 0xc2184e4120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3090 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc398f76bd0, 0xc398e9b4a0, 0xc398e9b500)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 2520 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x66, 0xc353feb100, 0xec, 0x100, 0xc3557f5800, 0xc8, 0x100, 0xc3564420c0, 0xc356442000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 53 [IO wait]:
net.(*pollDesc).Wait(0xc2080e2c30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e2c30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e2bd0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc212c24898)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20810e1a0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc20810e1a0, 0xc20810c528, 0xc20800f000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080f7140, 0xc20800f000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208111260)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208111260, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20810c4d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 54 [select]:
net/http.(*persistConn).writeLoop(0xc20810c4d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3232 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc393c26480, 0xc8, 0xc3a29ea700, 0xec, 0x100, 0xc3a1cb7800, 0xbb, 0x100, 0xc3a2df1140, 0xc3a2df1080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3282 [select]:
net/http.(*persistConn).roundTrip(0xc20810c630, 0xc26b5b3110, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc269bdb6c0, 0xc2b2b159d0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc269bdb6c0, 0x7fb4f7228fe8, 0xe94600, 0x61, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc269bdb6c0, 0x61, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc269bdb6c0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc269bdb6c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc269bdb6c0, 0x7fb4f722ca40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc2f7a06310, 0xc26b8900a0, 0x4c, 0x7fb4f722ca40, 0xc3a5cdea00, 0x9b6f00, 0xc2552a69c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc2e202dc48, 0xc26b8900a0, 0x4c, 0x9b6f00, 0xc2552a69c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Bucket).Refresh(0xc258307040, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:687 +0xf2
github.com/couchbase/go-couchbase.(*Pool).GetBucket(0xc32c057c70, 0xc2080d7910, 0x8, 0xc3a5da9340, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:820 +0x254
github.com/couchbase/goxdcr/utils.LocalBucket(0xc3a5ca7ab0, 0xe, 0xc2080d7910, 0x8, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:110 +0x1f3
github.com/couchbase/goxdcr/service_impl.(*ClusterInfoSvc).GetBucket(0xc2080380f0, 0x7fb4f722cf40, 0xc20801f440, 0xc2080d7910, 0x8, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/cluster_info_service.go:133 +0x25e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc393cea680, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:257 +0x115
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc393cea680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3a55e6120)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2984 [select]:
net/http.(*persistConn).roundTrip(0xc239c7e160, 0xc211f19680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc269bda1a0, 0xc269c6be00, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc269bda1a0, 0x7fb4f7228fe8, 0xc20802c090, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc269bda1a0, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc269bda1a0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc269bda1a0, 0xeb0d80, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/goxdcr/utils.doRestCall(0xc269bda1a0, 0x0, 0x818220, 0xc39d94e480, 0xeb0d80, 0xc20802b450, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc26b77a0e0, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:173 +0x201
github.com/couchbase/goxdcr/utils.QueryRestApi(0xc26b77a0e0, 0x15, 0x9e0430, 0xc, 0xc3944bd700, 0x9bf4c0, 0x3, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:134 +0x17a
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostInfo(0xc20801f440, 0x10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:113 +0x139
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:157 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostName(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:177 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyMemcachedAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:63 +0xa0
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyKVNodes(0xc20801f440, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:79 +0xa1
github.com/couchbase/goxdcr/pipeline_utils.GetSourceVBMapForReplication(0x7fb4f7229890, 0xc2080380f0, 0x7fb4f72298e0, 0xc20801f440, 0xc39335c380, 0xc3941dc020, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_utils/pipeline_utils.go:47 +0x240
github.com/couchbase/goxdcr/pipeline_svc.(*TopologyChangeDetectorSvc).validateSourceTopology(0xc3941ab440, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/topology_change_detector.go:163 +0x167
github.com/couchbase/goxdcr/pipeline_svc.(*TopologyChangeDetectorSvc).validate(0xc3941ab440, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/topology_change_detector.go:105 +0x36
github.com/couchbase/goxdcr/pipeline_svc.(*TopologyChangeDetectorSvc).watch(0xc3941ab440, 0xc3940b31f0, 0xc3941c2880)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/topology_change_detector.go:99 +0x1fa
created by github.com/couchbase/goxdcr/pipeline_svc.(*TopologyChangeDetectorSvc).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/topology_change_detector.go:65 +0x28d

goroutine 825 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea38c0, 0x12d, 0xc26e47c700, 0xec, 0x100, 0xc26e279000, 0xcf, 0x100, 0xc26e7920c0, 0xc26e792000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3037 [runnable]:
net/http.(*persistConn).roundTrip(0xc39dd37b80, 0xc210b88500, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc26b397c70, 0xc26b1fc1b0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc26b397c70, 0x7fb4f7228fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc26b397c70, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc26b397c70, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc26b397c70, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc26b397c70, 0x0, 0x818220, 0xc32c474840, 0xeb0d80, 0xc2080d7b10, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3941d3fb7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d6580, 0xd, 0xc2080d6590, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080d7b20, 0x9ed550, 0xe, 0xc32754d200, 0xeb0d80, 0x5, 0x66504c, 0x30000c32c474838, 0xc3941b0500, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080d7b20, 0xc3940b2690, 0xc26b1759c0, 0xc31b14be00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3940aa4b0, 0x2, 0xc3941f3790, 0x64, 0x98, 0xc394260bd0, 0xc393a50d00, 0xc39437ae70, 0xc398cc13e0, 0xc398cbe600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 1166 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x2, 0xc26c652800, 0xea, 0x100, 0xc286d97800, 0xcf, 0x100, 0xc2872f5ec0, 0xc2872f5e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2998 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f784b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3126 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc393c26480, 0xc393eb8230, 0xc393c26620)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 59 [runnable]:
net/http.(*persistConn).readLoop(0xc20802e580)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 60 [select]:
net/http.(*persistConn).writeLoop(0xc20802e580)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3128 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc393c26480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3097 [runnable]:
encoding/binary.bigEndian.PutUint64(0xc2673a8da8, 0x8, 0x10, 0x1)
	/usr/local/go/src/encoding/binary/binary.go:114
github.com/couchbase/goxdcr/parts.(*Router).ComposeMCRequest(0xc393c08a00, 0xc3141ad600, 0x44)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:102 +0x616
github.com/couchbase/goxdcr/parts.(*Router).route(0xc393c08a00, 0x8ed920, 0xc3141ad600, 0xc227b22380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:150 +0x924
github.com/couchbase/goxdcr/parts.*Router.(github.com/couchbase/goxdcr/parts.route)·fm(0x8ed920, 0xc3141ad600, 0xc2020c5727, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:69 +0x4d
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc393f68390, 0x8ed920, 0xc3141ad600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:57 +0xe9
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc393af2480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1716 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x5, 0xc2d52e4500, 0xea, 0x100, 0xc2d5054400, 0x6f, 0x80, 0xc2d543cfc0, 0xc2d543cf00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3228 [runnable]:
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc2b6dfa480, 0x7fb4f722ac00, 0xc39ff79550, 0xc3a270d720, 0x18, 0x18, 0x0, 0x7fb4f722ac00, 0xc3a208dbe0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:147
github.com/couchbase/gomemcached/client.getResponse(0x7fb4f722ac00, 0xc39ff79550, 0xc3a270d720, 0x18, 0x18, 0xc2b6dfa480, 0x8f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc3a282a1e0, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x25d
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc3a282a1e0, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc39fbb7040, 0x2, 0x9cac80, 0x3, 0xc3a1aa3b60)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 1825 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0x12d, 0xc2eb890800, 0xec, 0x100, 0xc2eb495000, 0xcf, 0x100, 0xc2ebcb84e0, 0xc2ebcb8420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1554 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x0, 0xc246e6c400, 0xea, 0x100, 0xc2c673a800, 0xb8, 0x100, 0xc2c924c660, 0xc2c924c5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1236 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0x130, 0xc2083b3a00, 0xec, 0x100, 0xc2a342e800, 0xbc, 0x100, 0xc2a37c5560, 0xc2a37c54a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2150 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980240, 0x12c, 0xc2d3f37700, 0xec, 0x100, 0xc28edb2000, 0xc4, 0x100, 0xc2d74ded80, 0xc2d74decc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1913 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x69, 0xc31a642100, 0xec, 0x100, 0xc31b07c000, 0xba, 0x100, 0xc31af97b60, 0xc31af97aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 719 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea38c0, 0xc8, 0xc246e6d600, 0xec, 0x100, 0xc247474000, 0xbb, 0x100, 0xc24773a480, 0xc24773a3c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 717 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x67, 0xc2465e1000, 0xec, 0x100, 0xc2463d3000, 0xb3, 0x100, 0xc2469c0de0, 0xc2469c0d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1186 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0xc9, 0xc26c653e00, 0xec, 0x100, 0xc2886bb000, 0xbd, 0x100, 0xc288d43f80, 0xc288d43ec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2552 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x65, 0xc36e5a6d00, 0xec, 0x100, 0xc371a4f000, 0xc6, 0x100, 0xc372296480, 0xc3722963c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1922 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x69, 0xc31a642400, 0xec, 0x100, 0xc319ed3800, 0xba, 0x100, 0xc31a80a240, 0xc31a80a180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1895 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x3, 0xc317d42100, 0xea, 0x100, 0xc31695f000, 0xb5, 0x100, 0xc3181bf500, 0xc3181bf440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1190 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x64, 0xc248387e00, 0xec, 0x100, 0xc292b33800, 0xc9, 0x100, 0xc2931a68a0, 0xc2931a67e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2192 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980240, 0xca, 0xc26e47d700, 0xec, 0x100, 0xc2adff1000, 0xbc, 0x100, 0xc2d41ff380, 0xc2d41ff2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1173 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x66, 0xc27956d900, 0xec, 0x100, 0xc26bee8800, 0xc8, 0x100, 0xc26d4a4cc0, 0xc26d4a4c00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3003 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc393583ea0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 2543 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0xcd, 0xc373514300, 0xec, 0x100, 0xc372a95000, 0xba, 0x100, 0xc37369c060, 0xc37355df80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3326 [select]:
net/http.(*persistConn).writeLoop(0xc318787080)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3308 [runnable]:
net/http.(*persistConn).readLoop(0xc3185408f0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1150 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0x12c, 0xc279a97a00, 0xec, 0x100, 0xc27bee8800, 0xc4, 0x100, 0xc25289d1a0, 0xc25289d0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2220 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980240, 0x12d, 0xc2678ede00, 0xec, 0x100, 0xc3292a3800, 0xcf, 0x100, 0xc3297793e0, 0xc329779320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 126 [select]:
net/http.(*persistConn).writeLoop(0xc20810c630)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 125 [runnable]:
net.(*pollDesc).Wait(0xc2080e3870, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e3870, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e3810, 0xc208376000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc210ae0dc8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20810e0d8, 0xc208376000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc20810e0d8, 0xc20810c688, 0xc208376000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208367dc0, 0xc208376000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20813e6c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20813e6c0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20810c630)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 813 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x65, 0xc26cbb8400, 0xec, 0x100, 0xc26cf98000, 0xc6, 0x100, 0xc26d0425a0, 0xc26d0424e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 145 [syscall, 2 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 1454 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0x12c, 0xc2bae14000, 0xec, 0x100, 0xc2ba33e800, 0xc4, 0x100, 0xc2bad1db60, 0xc2bad1daa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1158 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0xc8, 0xc26ef98300, 0xec, 0x100, 0xc25c2a8000, 0xad, 0x100, 0xc25c2ec7e0, 0xc25c2ec720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1851 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0xcb, 0xc2f785e300, 0xec, 0x100, 0xc2f6feb000, 0xc9, 0x100, 0xc2f7aa9ec0, 0xc2f7aa9e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 465 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x64, 0xc218cc4d00, 0xec, 0x100, 0xc21b315000, 0xc9, 0x100, 0xc212a89320, 0xc212a89260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 392 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xc8, 0xc2126cbc00, 0xec, 0x100, 0xc213003800, 0xb9, 0x100, 0xc213744180, 0xc2137440c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1357 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0x12f, 0xc2ad7d9b00, 0xec, 0x100, 0xc2ae704400, 0x52, 0x80, 0xc2ae99c960, 0xc2ae99c8a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2503 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0x12c, 0xc350458c00, 0xec, 0x100, 0xc3542b4000, 0xb4, 0x100, 0xc3542c3380, 0xc3542c32c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1512 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x2, 0xc2795be100, 0xea, 0x100, 0xc2a3681000, 0xc0, 0x100, 0xc292b92cc0, 0xc292b92c00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 453 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x65, 0xc2083b3300, 0xec, 0x100, 0xc21b313800, 0xc6, 0x100, 0xc21debe6c0, 0xc21debe600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 428 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x2, 0xc2083b2a00, 0xea, 0x100, 0xc2137b6000, 0xa7, 0x100, 0xc215160c60, 0xc215160ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3325 [runnable]:
net/http.(*persistConn).readLoop(0xc318787080)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 432 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xca, 0xc2083b3100, 0xec, 0x100, 0xc21abf3800, 0xbc, 0x100, 0xc21b1a0b40, 0xc21b1a0a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1011 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0xc8, 0xc279a96a00, 0xeb, 0x100, 0xc279d53800, 0x1d, 0x20, 0xc279dc09c0, 0xc279dc0900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3036 [select]:
net/http.(*persistConn).roundTrip(0xc3185408f0, 0xc2d951dcc0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc25ea091e0, 0xc264c3ade0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc25ea091e0, 0x7fb4f7228fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc25ea091e0, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc25ea091e0, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc25ea091e0, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc25ea091e0, 0x0, 0x818220, 0xc39d94fa28, 0xeb0d80, 0xc2080d7b10, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3941d3fb7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d6580, 0xd, 0xc2080d6590, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080d7b20, 0x9ed550, 0xe, 0xc2b6dfa4e0, 0xeb0d80, 0x5, 0x66504c, 0x30000c39d94fa20, 0xc3941a6c00, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080d7b20, 0xc3940b2690, 0xc267372220, 0xc2080b9e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3940aa4b0, 0x1, 0xc3941f36c8, 0x64, 0xfc, 0xc394260bd0, 0xc393a50d00, 0xc39437ae70, 0xc398cc13e0, 0xc398cbe600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 745 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea38c0, 0xc9, 0xc246e6ce00, 0xec, 0x100, 0xc24f6a3000, 0xbd, 0x100, 0xc2310071a0, 0xc231006d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 477 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xcb, 0xc2126cac00, 0xec, 0x100, 0xc22de80000, 0xc9, 0x100, 0xc22dee14a0, 0xc22dee13e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1544 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0xc8, 0xc2678edc00, 0xec, 0x100, 0xc2c1ada800, 0xbb, 0x100, 0xc2c2406a20, 0xc2c2406960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3096 [select]:
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc394257290, 0xc398e9bc20)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:547 +0xa0e
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3324 [select]:
net/http.(*persistConn).writeLoop(0xc318786fd0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2533 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x1, 0xc3622a2800, 0xea, 0x100, 0xc361905800, 0xc7, 0x100, 0xc3622f3140, 0xc3622f3080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2944 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f78870)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 447 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x1, 0xc2372b0000, 0xea, 0x100, 0xc235d60800, 0xc7, 0x100, 0xc23721baa0, 0xc23721b9e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1192 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0x12d, 0xc2465e0d00, 0xec, 0x100, 0xc294c5f800, 0xcf, 0x100, 0xc294ea0a80, 0xc294ea09c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1117 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436240, 0xca, 0xc27c65ac00, 0xec, 0x100, 0xc27d40f000, 0xbc, 0x100, 0xc27d5e58c0, 0xc27d5e5800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3307 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc393c26240, 0x0, 0xc3a166cb00, 0xea, 0x100, 0xc315f4d800, 0xb8, 0x100, 0xc2de1851a0, 0xc2de1850e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 799 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea38c0, 0xcb, 0xc26c653500, 0xec, 0x100, 0xc26bcc3000, 0xc9, 0x100, 0xc26c8a8cc0, 0xc26c8a8c00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 771 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea38c0, 0x12c, 0xc2126cb800, 0xec, 0x100, 0xc258621000, 0xb4, 0x100, 0xc2591ed2c0, 0xc2591ed200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1843 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x4, 0xc2eb891b00, 0xea, 0x100, 0xc2ed0d9800, 0xa2, 0x100, 0xc2ed103f20, 0xc2ed103e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2800 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc37d8eeb40, 0x64, 0xc386f69700, 0xec, 0x100, 0xc37d763000, 0xc7, 0x100, 0xc387794660, 0xc3877945a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 409 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x12c, 0xc218cc4100, 0xec, 0x100, 0xc218c4c000, 0xc4, 0x100, 0xc218ce23c0, 0xc218ce2300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3323 [IO wait]:
net.(*pollDesc).Wait(0xc2b80fed80, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2b80fed80, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2b80fed20, 0xc26f411000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc20c62fd20)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39d94e398, 0xc26f411000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc39d94e398, 0xc318787028, 0xc26f411000, 0x1000, 0x1000, 0x7fb4f72174c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc265ce1600, 0xc26f411000, 0x1000, 0x1000, 0xc1eea6b1b8, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc258b41560)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc258b41560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc318786fd0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1433 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x1, 0xc2b50ee100, 0xea, 0x100, 0xc2b4c27000, 0xc7, 0x100, 0xc2b5126f60, 0xc2b5126ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 989 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x0, 0xc2795bf200, 0xe9, 0x100, 0xc2798b4c00, 0x13, 0x20, 0xc27982f560, 0xc27982f4a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2990 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc393c26240, 0xc393eb8070, 0xc393c263e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 1476 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x66, 0xc2bb4a3900, 0xec, 0x100, 0xc26bba9800, 0xc8, 0x100, 0xc249c61140, 0xc249c61080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1498 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0x12e, 0xc2ad7d9300, 0xec, 0x100, 0xc2b3102800, 0xb3, 0x100, 0xc254b55a40, 0xc254b55980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 747 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x0, 0xc2372b1200, 0xea, 0x100, 0xc24cac3800, 0xb8, 0x100, 0xc218d52720, 0xc218d52660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 838 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x1, 0xc26ef98800, 0xea, 0x100, 0xc26ef3e000, 0xc7, 0x100, 0xc26effc0c0, 0xc26effc000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 618 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x64, 0xc208162700, 0xec, 0x100, 0xc230874000, 0x47, 0x80, 0xc232408a20, 0xc232432660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3082 [select]:
net/http.(*persistConn).writeLoop(0xc398d13e40)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2942 [semacquire]:
sync.(*WaitGroup).Wait(0xc398cc13e0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc3940aa4b0, 0xc20800b700, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc2080323c0, 0x7fb4f72325d0, 0xc3939c16b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7fb4f72325d0, 0xc3939c16b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 1799 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x6a, 0xc2dc33a500, 0xec, 0x100, 0xc2dd293800, 0xcc, 0x100, 0xc2dddac0c0, 0xc2dddac000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2941 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc20806ed20, 0x7fb4f722c870, 0xc3a3653040, 0xc26377cea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038100, 0x7fb4f722c870, 0xc3a3653040, 0xc26377cea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fb4f722c870, 0xc3a3653040, 0xc26377cea0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc20806ed80, 0x7fb4f722c870, 0xc3a3653040, 0xc26377cea0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc39362d400)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 1869 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0x12e, 0xc30a39d300, 0xec, 0x100, 0xc30a2ff800, 0x9b, 0x100, 0xc30a5f59e0, 0xc30a5f5920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3038 [select]:
net/http.(*persistConn).roundTrip(0xc31833c6e0, 0xc2116188b0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc26b54ab60, 0xc26b556390, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc26b54ab60, 0x7fb4f7228fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc26b54ab60, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc26b54ab60, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc26b54ab60, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc26b54ab60, 0x0, 0x818220, 0xc32c474738, 0xeb0d80, 0xc2080d7b10, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3941d3fb7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d6580, 0xd, 0xc2080d6590, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc2080d7b20, 0x9ed550, 0xe, 0xc26b520240, 0xeb0d80, 0x5, 0x66504c, 0x30000c32c474730, 0xc3941b1e00, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc2080d7b20, 0xc3940b2690, 0xc26b566080, 0xc31b14de00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3940aa4b0, 0x3, 0xc3941f3858, 0x29, 0x34, 0xc394260bd0, 0xc393a50d00, 0xc39437ae70, 0xc398cc13e0, 0xc398cbe600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 535 [select]:
net/http.(*persistConn).writeLoop(0xc239c7e160)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 534 [runnable]:
net.(*pollDesc).Wait(0xc23988de90, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc23988de90, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc23988de30, 0xc215b40000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc210ae0d70)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc218f750c0, 0xc215b40000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc218f750c0, 0xc239c7e1b8, 0xc215b40000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc236dd8860, 0xc215b40000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc238802600)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc238802600, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc239c7e160)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2083 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x64, 0xc3154cc000, 0xec, 0x100, 0xc3121bb000, 0xad, 0x100, 0xc30c641e60, 0xc30c641da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1783 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0xc8, 0xc2daa5d900, 0xec, 0x100, 0xc2db4f9000, 0xbb, 0x100, 0xc2dba3afc0, 0xc2dba3af00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2524 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0x12d, 0xc3610a0f00, 0xec, 0x100, 0xc3619f6000, 0xcf, 0x100, 0xc361aa1140, 0xc361aa1080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1552 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0x12d, 0xc2372b1e00, 0xec, 0x100, 0xc2c98d5000, 0xcf, 0x100, 0xc2ca0b79e0, 0xc2ca0b7920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2992 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc393c26240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2175 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x67, 0xc2795bfe00, 0xec, 0x100, 0xc304abb800, 0xb3, 0x100, 0xc2172c5b00, 0xc2172c5a40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1490 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0xca, 0xc27c443800, 0xec, 0x100, 0xc2a342d000, 0xbc, 0x100, 0xc2b7c40720, 0xc2b7c40660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1538 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x67, 0xc26ef99e00, 0xec, 0x100, 0xc287fbe800, 0xb3, 0x100, 0xc29376fec0, 0xc29376fe00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 777 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x66, 0xc208055800, 0xec, 0x100, 0xc25bbec000, 0xc8, 0x100, 0xc25bc33c80, 0xc25bc33bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3171 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc393af2780)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 803 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea3680, 0x2, 0xc266fc0f00, 0xea, 0x100, 0xc266e7f800, 0xcf, 0x100, 0xc2673615c0, 0xc267361500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1215 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x3, 0xc21826de00, 0xea, 0x100, 0xc2a2c59800, 0xb5, 0x100, 0xc2a303ef00, 0xc2a303ee40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 785 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc21cea38c0, 0xca, 0xc260debb00, 0xec, 0x100, 0xc263d43000, 0xbc, 0x100, 0xc26438bce0, 0xc26438bc20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3311 [select]:
net/http.(*persistConn).writeLoop(0xc31833c6e0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3035 [chan send]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).UpdateSettings(0xc393af2480, 0xc3941d3410, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:628 +0x2b7
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).UpdateSettings(0xc3939c16b0, 0xc3941d3410, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:633 +0x2b0
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).setTimestampForVB(0xc3940aa4b0, 0xc, 0xc264c3bad0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:392 +0x9b8
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3940aa4b0, 0x0, 0xc3941f3600, 0x64, 0x160, 0xc394260bd0, 0xc393a50d00, 0xc39437ae70, 0xc398cc13e0, 0xc398cbe600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:460 +0x90b
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 2569 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x64, 0xc376914e00, 0xec, 0x100, 0xc379b6a000, 0xc9, 0x100, 0xc379c28780, 0xc379c286c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1904 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0x12c, 0xc319d1ad00, 0xec, 0x100, 0xc319ed2000, 0xc4, 0x100, 0xc319f606c0, 0xc319f60600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1855 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x2, 0xc2faf36800, 0xea, 0x100, 0xc2faa05000, 0xcf, 0x100, 0xc2fb172cc0, 0xc2fb172c00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1522 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876fc0, 0xc9, 0xc26feab100, 0xec, 0x100, 0xc28a532000, 0xbd, 0x100, 0xc289cf28a0, 0xc289cf27e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3322 [select]:
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc3a2ca4200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:557 +0x48b
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 1881 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0xca, 0xc3154cc500, 0xec, 0x100, 0xc314afb800, 0xbc, 0x100, 0xc315894d20, 0xc315894c60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1146 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x1, 0xc27c443300, 0xea, 0x100, 0xc27db0a000, 0xc7, 0x100, 0xc25a157080, 0xc25a156fc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1494 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x65, 0xc2b50ef700, 0xec, 0x100, 0xc2b8500000, 0xc6, 0x100, 0xc28568bda0, 0xc28568bce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1092 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc270436000, 0x0, 0xc27c442500, 0xea, 0x100, 0xc27b837000, 0xad, 0x100, 0xc27c53f620, 0xc27c53f560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3081 [IO wait]:
net.(*pollDesc).Wait(0xc398f8b020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc398f8b020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc398f8afc0, 0xc39900f000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc39902e298)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39392db08, 0xc39900f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc39392db08, 0xc398d13e98, 0xc39900f000, 0x1000, 0x1000, 0x7fb4f7217010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc39901f400, 0xc39900f000, 0x1000, 0x1000, 0xc1ef46d24e, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc398ddb9e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc398ddb9e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc398d13e40)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1366 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a4876d80, 0x2, 0xc2a91c5800, 0xe9, 0x100, 0xc2adcdc000, 0x1d, 0x20, 0xc2adcf4420, 0xc2adcf4360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1690 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacefb00, 0x12e, 0xc2d3f36d00, 0xeb, 0x100, 0xc2d429ac00, 0x27, 0x40, 0xc2d42ed440, 0xc2d42ed380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3173 [runnable]:
encoding/binary.bigEndian.Uint64(0xc32b91a200, 0x8, 0xabc, 0xa79)
	/usr/local/go/src/encoding/binary/binary.go:109
github.com/couchbase/gomemcached/client.makeUprEvent(0x57, 0x14b31f7a59bb0000, 0xd0eaf100d0, 0xc32b91a200, 0x1f, 0xabc, 0xc32b91a21f, 0x24, 0xa9d, 0xc32b91a243, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:111 +0x158
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc39d7d8510, 0xc39d8f8de0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:476 +0x1cc5
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 2137 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x0, 0xc2bb4a2900, 0xea, 0x100, 0xc29e545000, 0xb8, 0x100, 0xc2defd4480, 0xc2defd43c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1871 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2cacef8c0, 0x64, 0xc30a39d700, 0xec, 0x100, 0xc30d373800, 0xc9, 0x100, 0xc30d8d53e0, 0xc30d8d5320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2945 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f791d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2931 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc37d8eed80, 0x12d, 0xc392e86000, 0xec, 0x100, 0xc391a8e800, 0xcf, 0x100, 0xc392bdfa40, 0xc392bdf980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2571 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0x12e, 0xc37a28aa00, 0xec, 0x100, 0xc379b6b800, 0xb3, 0x100, 0xc37a2c83c0, 0xc37a2c8300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2499 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0xc9, 0xc344559500, 0xec, 0x100, 0xc348572000, 0xbd, 0x100, 0xc34861a180, 0xc34861a0c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2594 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x6b, 0xc37b6fd800, 0xec, 0x100, 0xc37b59f000, 0xbd, 0x100, 0xc37b95f9e0, 0xc37b95f920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2997 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f79540)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2203 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x65, 0xc26ef99f00, 0xec, 0x100, 0xc29c358800, 0xc6, 0x100, 0xc289f891a0, 0xc289f890e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2195 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980240, 0xc9, 0xc27956df00, 0xec, 0x100, 0xc2ff0af000, 0xdb, 0x100, 0xc2c69b6240, 0xc2c69b6180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3151 [runnable]:
github.com/couchbase/gomemcached/client.sendCommands(0xc39d8c4570, 0xc39d8c2840, 0xc39d8c28a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 2399 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x0, 0xc3359edd00, 0xea, 0x100, 0xc335f30000, 0x45, 0x80, 0xc335e94d20, 0xc335e94c60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3174 [runnable]:
github.com/couchbase/goxdcr/parts.(*Router).route(0xc3939e1f90, 0x8ed920, 0xc314dbf600, 0xc227b22380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:151 +0x9c9
github.com/couchbase/goxdcr/parts.*Router.(github.com/couchbase/goxdcr/parts.route)·fm(0x8ed920, 0xc314dbf600, 0xc201efd519, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/router.go:69 +0x4d
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc393f68510, 0x8ed920, 0xc314dbf600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:57 +0xe9
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc393af2780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 2428 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0xc8, 0xc33ba11300, 0xec, 0x100, 0xc33bad1000, 0xbb, 0x100, 0xc33c1dc6c0, 0xc33c1dc600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3105 [IO wait]:
net.(*pollDesc).Wait(0xc39d407950, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc39d407950, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc39d4078f0, 0xc2adb69000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc234272ce0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39392dd08, 0xc2adb69000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc3942916c8, 0xc2adb69000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc39d456540, 0xc2adb69000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc214a672c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc214a672c0, 0xc3a613200a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc214a672c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc3a62d8720, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc3a62d8720, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc214a672c0, 0xc2510a85b0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc394291680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc394291680)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2236 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980240, 0xc8, 0xc2372b0b00, 0xec, 0x100, 0xc32abe2800, 0xbb, 0x100, 0xc32b5b7f80, 0xc32b5b7ec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2156 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x64, 0xc2b0e45100, 0xec, 0x100, 0xc30d84a000, 0xc3, 0x100, 0xc2898b2240, 0xc2898b2180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2980 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc393583eb0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3200 [select]:
net/http.(*persistConn).writeLoop(0xc39dd37b80)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2234 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x2, 0xc2465e1500, 0xea, 0x100, 0xc32abe1000, 0xcf, 0x100, 0xc32af6c600, 0xc32af6c540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2165 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980240, 0x12e, 0xc27c442100, 0xec, 0x100, 0xc30c45d000, 0xb4, 0x100, 0xc2bdd7b2c0, 0xc2bdd7b200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3127 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc393c26480, 0xa900c8, 0xc32c73fb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:836 +0x14d1
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc393c26480, 0xc39f2500c8, 0xc32c73fb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc393c26480, 0xc39d3f8310, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc393c26480, 0xc39d3f8310, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc393c26480, 0xc393eb8150, 0xc393c26620, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 2991 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc393c26240, 0xa90000, 0xc39d941b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:836 +0x14d1
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc393c26240, 0xc398cd0000, 0xc39d941b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc393c26240, 0xc398f8b490, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc393c26240, 0xc398f8b490, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc393c26240, 0xc393aa3f80, 0xc393c263e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 2218 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc318980000, 0x1, 0xc2678ece00, 0xea, 0x100, 0xc325fb0800, 0xc7, 0x100, 0xc3274a0840, 0xc3274a0780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3309 [runnable]:
net/http.(*persistConn).writeLoop(0xc3185408f0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3306 [runnable]:
net.(*pollDesc).Wait(0xc3a1b03720, 0x77, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitWrite(0xc3a1b03720, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:93 +0x43
net.(*netFD).Write(0xc3a1b036c0, 0xc3a3691400, 0xc9a, 0xd00, 0xc52, 0x7fb4f7228df0, 0xc2dc04fca8)
	/usr/local/go/src/net/fd_unix.go:335 +0x5ee
net.(*conn).Write(0xc39ff79548, 0xc3a3691400, 0xc9a, 0xd00, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:129 +0xdc
github.com/couchbase/goxdcr/parts.(*CapiNozzle).tcpProxy(0xc393c26240, 0x0, 0xc2de1851a0, 0xc2c76f69a0, 0xc2de1850e0, 0xc2de185140)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:909 +0x492
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:831 +0xc91

goroutine 3328 [runnable]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc393c26480, 0xc8, 0xc2678ec400, 0xec, 0x100, 0xc3292a5000, 0xbb, 0x100, 0xc2bf57ae40, 0xc2bf57ad80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2994 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f79360)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3098 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc393af2480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 3001 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f79720)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3161 [runnable]:
net.(*pollDesc).Wait(0xc39d5385a0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc39d5385a0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc39d538540, 0xc39d74d000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc263a6fca8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39392ddd8, 0xc39d74d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fb4f722ac00, 0xc39392ddd8, 0xc39d45c318, 0xc39d74d000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc39d773f80, 0xc39d74d000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc39d6309c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc39d6309c0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc39d45c2c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2899 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc37d8eeb40, 0x1, 0xc38ea9d300, 0xea, 0x100, 0xc38e18f800, 0xc7, 0x100, 0xc38effca20, 0xc38effc960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2475 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x0, 0xc33d256700, 0xea, 0x100, 0xc33d3bc000, 0xa3, 0x100, 0xc33d351ec0, 0xc33d351e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2586 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0xd0, 0xc37bfd2500, 0xec, 0x100, 0xc37bd80800, 0xb2, 0x100, 0xc37c21af60, 0xc37c21aea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2397 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c240, 0x12c, 0xc3359ec700, 0xec, 0x100, 0xc335b5cc00, 0x1e, 0x20, 0xc335b963c0, 0xc335b96300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2490 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32ca6c000, 0x2, 0xc348c27a00, 0xea, 0x100, 0xc349462000, 0xcf, 0x100, 0xc3495fcba0, 0xc3495fcae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3310 [runnable]:
net/http.(*persistConn).readLoop(0xc31833c6e0)
	/usr/local/go/src/net/http/transport.go:858 +0x36d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3172 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc393af2780, 0xc39d4521e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 3223 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc393c26240, 0x0, 0xc39fe6a600, 0xea, 0x100, 0xc398cc6800, 0xb8, 0x100, 0xc3a05ec0c0, 0xc3a05ec000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3094 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc393af2480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3162 [select]:
net/http.(*persistConn).writeLoop(0xc39d45c2c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3002 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f78ff0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3175 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc393af2780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 2982 [select]:
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).checkpointing(0xc3940aa4b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:636 +0x419
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:196 +0x292

goroutine 2934 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc37d8eeb40, 0x66, 0xc3929b0800, 0xec, 0x100, 0xc393052000, 0xc8, 0x100, 0xc3931a99e0, 0xc3931a9920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3000 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f78a50)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2995 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f78e10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2996 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f798b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3199 [select]:
net/http.(*persistConn).readLoop(0xc39dd37b80)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2943 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f78690)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2999 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc393f78c30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3150 [chan receive]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc39fbb7040, 0x9cac80, 0x3, 0xc2080d7910)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc393cea680, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc393cea680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc39d8c23c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2870 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc37d8eed80, 0x12f, 0xc38a71dc00, 0xec, 0x100, 0xc38e999800, 0xa7, 0x100, 0xc38f030ba0, 0xc38f030ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3292 [IO wait]:
net.(*pollDesc).Wait(0xc2f7a064c0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2f7a064c0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2f7a06460, 0xc2d49fe000, 0x1000, 0x1000, 0x0, 0x7fb4f7228df0, 0xc210ae0e80)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3a31f8000, 0xc2d49fe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc39f546048, 0xc2d49fe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc2523a6c80, 0xc2d49fe000, 0x1000, 0x1000, 0x1, 0x0, 0x0)
	/usr/local/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc3003260c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc3003260c0, 0xc31ae0fb0a, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc3003260c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc2637e56e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc2637e56e0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc3003260c0, 0xc261b5a1a0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc39f546000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc39f546000)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e
[goport] 2017/04/25 13:33:45 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:33:46.245-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:33:46.246-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:33:46.246-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:33:46.267-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:33:46.271-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:33:46.271-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:33:46.271-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:33:46.271-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:33:46.271-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:33:46.271-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:33:46.271-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:33:46.271-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:33:46.271-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:33:46.271-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:33:46.271-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:33:46.271-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:33:46.271-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:33:46.271-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:33:46.271-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:33:46.271-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:33:46.271-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:33:46.272-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:33:46.272-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:33:46.272-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:33:46.275-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:33:46.275-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:33:46.276-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:33:46.276-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:33:46.395-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:34:01.322-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:34:01.326-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:34:01.434-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:01.434-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:34:01.435-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:34:01.491-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:01.492-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:34:01.492-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:01.492-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:34:01.492-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:01.492-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:34:01.492-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:34:01.495-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:34:01.498-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
GenericSupervisor 2017-04-25T13:34:01.500-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:34:01.500-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
GenericSupervisor 2017-04-25T13:34:01.500-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:01.500-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:01.500-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:01.500-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:01.500-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:34:01.500-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
CheckpointManager 2017-04-25T13:34:01.500-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
CapiUtils 2017-04-25T13:34:01.536-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:01.536-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:01.536-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:34:01.536-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
StatisticsManager 2017-04-25T13:34:01.537-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:34:01.537-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
XDCRFactory 2017-04-25T13:34:01.537-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:34:01.537-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327 is constructed, start it
GenericPipeline 2017-04-25T13:34:01.538-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[209 246 194 321 330 182 247 191 286 299 185 288 187 243 256 301 254 311 215 212 175 214 289 307 198 210 218 178 208 190 252 188 189 239 262 283 202 287 326 224 261 274 320 337 206 227 244 271 285 221 248 253 312 266 278 333 235 263 315 228 237 250 290 176 181 324 325 255 284 204 222 267 316 195 177 201 268 282 258 203 305 245 279 251 334 260 273 234 236 296 297 303 184 193 207 238 205 335 173 242 264 322 275 319 179 328 180 269 219 231 338 318 331 336 291 313 339 200 216 183 211 229 295 304 172 280 265 314 223 249 292 276 281 329 196 230 233 293 171 213 220 298 257 332 340 272 309 199 217 197 232 310 186 192 241 174 225 302 226 259 317 308 270 327 277 294 300 306 240 323] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[87 140 165 170 17 31 60 76 118 49 59 90 110 5 123 112 41 51 79 96 109 6 50 70 97 168 20 58 103 160 92 156 126 73 99 45 72 127 88 158 132 113 152 119 125 141 66 94 129 26 27 34 52 47 21 37 55 84 48 75 143 144 16 86 114 154 151 159 29 128 131 138 2 83 105 150 30 46 74 78 3 38 130 39 98 10 145 133 136 163 106 134 167 137 142 56 62 95 121 100 24 135 148 11 32 15 36 9 57 115 67 157 162 1 139 101 40 107 117 44 81 82 169 7 8 14 28 12 25 64 89 161 166 147 149 18 71 116 80 124 63 68 85 155 43 22 33 53 153 35 42 65 54 69 93 61 111 4 104 19 23 122 108 77 91 102 13 146 0 120 164]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[116 18 71 80 124 63 155 68 85 43 53 153 22 33 65 35 42 93 54 69 61 111 4 104 122 19 23 108 102 77 91 13 146 164 0 120 60 76 87 140 165 170 17 31 90 110 118 49 59 5 123 79 96 112 41 51 70 97 109 6 50 103 160 168 20 58 92 156 126 73 99 127 45 72 88 158 132 113 152 141 119 125 34 52 66 94 129 26 27 47 55 84 21 37 143 144 48 75 114 154 16 86 131 138 151 159 29 128 105 2 83 74 78 150 30 46 130 3 38 39 98 10 145 133 136 163 167 106 134 95 121 137 142 56 62 100 148 24 135 11 32 15 36 115 9 57 162 67 157 1 139 101 117 40 107 14 28 44 81 82 169 7 8 12 89 161 25 64 166 147 149] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[223 265 314 249 292 230 233 276 281 329 196 171 293 220 298 213 257 332 340 199 272 309 197 217 186 192 232 310 174 225 241 226 302 259 308 317 270 327 240 277 294 300 306 323 209 246 194 321 330 182 247 191 286 299 185 288 187 243 301 256 254 311 212 215 175 289 307 214 178 198 210 218 208 252 190 188 189 239 262 283 202 287 326 224 261 274 320 337 206 221 227 244 271 285 248 253 312 235 266 278 333 263 315 176 181 228 237 250 290 324 325 255 284 222 204 195 267 316 177 201 268 282 258 203 245 279 305 251 334 234 236 260 273 207 238 296 297 303 184 193 173 205 335 322 242 264 179 275 319 180 269 328 231 338 219 291 318 331 336 313 339 216 200 183 211 229 172 280 295 304]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[active:true worker_batch_size:500 doc_batch_size_kb:2048 log_level:Info target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:34:01.538-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327...
AsyncListener 2017-04-25T13:34:01.549-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:01.549-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:34:01.549-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:01.549-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:01.550-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:34:01.550-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:01.550-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:34:01.550-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:34:01.550-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:34:01.550-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:34:01.550-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:34:01.550-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:34:01.550-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:34:01.550-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:34:01.552-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:34:01.552-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:34:01.552-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:34:01.552-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:34:01.552-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:34:01.726-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:01.726-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:01.727-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:34:01.737-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:34:01.737-05:00 [INFO] connectionSize=5
ToplogyChangeDetector 2017-04-25T13:34:01.740-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:34:01.741-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:34:01.744-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
TCPConnPoolMgr 2017-04-25T13:34:01.746-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:34:01.746-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:34:01.746-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:34:01.747-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:34:01.747-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:34:01.747-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:34:01.803-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:34:01.814-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:34:01.814-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:01.814-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:34:01.814-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:34:01.814-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:34:01.816-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:34:01.925-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:34:01.925-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:34:01.926-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:34:01.941-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:01.941-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:02.106-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:34:02.106-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:02.106-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:34:02.107-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
CheckpointManager 2017-04-25T13:34:02.107-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:34:02.108-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:34:02.108-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
DcpNozzle 2017-04-25T13:34:02.107-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CheckpointManager 2017-04-25T13:34:02.108-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:34:02.108-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:34:02.108-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327 is started----------
PipelineManager 2017-04-25T13:34:02.108-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:34:02.108-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:34:02.108-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:34:02 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:34:02.275-05:00 [INFO] Send batch count=500
2017/04/25 13:34:02 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 28 successful
CapiNozzle 2017-04-25T13:34:02.435-05:00 [INFO] Send batch count=2
2017/04/25 13:34:02 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 263 successful
StatisticsManager 2017-04-25T13:34:02.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:34:02 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 58 successful
CapiNozzle 2017-04-25T13:34:02.646-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:02.719-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:34:02 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 170 successful
CapiNozzle 2017-04-25T13:34:02.747-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:34:02 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 72 successful
CapiNozzle 2017-04-25T13:34:02.919-05:00 [ERROR] Received malformed response from tcp connection vb=202. err=malformed HTTP response "0_009_00774_136_16-3.3F\"}]\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", num_bytes=26, res_buf={0_009_00774_136_16-3.3F"}]}

CapiNozzle 2017-04-25T13:34:02.919-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err Received malformed response from tcp connection vb=202. err=malformed HTTP response "0_009_00774_136_16-3.3F\"}]\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", num_bytes=26, res_buf={0_009_00774_136_16-3.3F"}]}
.
2017/04/25 13:34:02 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:34:02 UPR_STREAMREQ for vb 73 successful
CapiNozzle 2017-04-25T13:34:02.943-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
2017/04/25 13:34:03 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 194 successful
CheckpointManager 2017-04-25T13:34:03.313-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
CheckpointManager 2017-04-25T13:34:03.314-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:34:03 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:34:03 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:34:03.412-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:34:03.422-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:34:03.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:34:03.727-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:03.746-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:34:03.872-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:34:03.887-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:34:04.099-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:04.112-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:34:04.269-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:34:04.323-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:34:04.487-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:04.487-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:34:04.487-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:34:04.487-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:34:04.487-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:34:04.495-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:34:04.496-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:34:04.498-05:00 [INFO] err_list=[{"time":"2017-04-25T13:34:04.498324766-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:34:04.498-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:34:04.553-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:34:04.553-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327 {"Errors": "[{\"time\":\"2017-04-25T13:34:04.498324766-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 503, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 628, "docs_filtered": 0, "docs_opt_repd": 2, "docs_processed": 4, "docs_received_from_dcp": 20911, "docs_rep_queue": 0, "docs_written": 2, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 2, "set_failed_cr_source": 628, "set_filtered": 0, "set_received_from_dcp": 20911, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:34:04.592-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:04.592-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:04.604-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:34:04.646-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:34:04.646-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:34:04.646-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:34:04.646-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:04.646-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:04.728-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:04.730-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:04.741-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:34:05.140-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:05.161-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:34:05.188-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:34:05.204-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:34:05.568-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:05.585-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:34:06.016-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:34:06.027-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:34:06.167-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:06.168-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:34:06.168-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:34:06.168-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:06.168-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:06.278-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:06.278-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:06.290-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:34:06.650-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:34:06.650-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:34:06.650-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:34:06.650-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:06.650-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:06.757-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:06.787-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:34:06.816-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:06.816-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:06.838-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:34:07.215-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:07.235-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:34:07.724-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:34:07.758-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:34:07.799-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:07.799-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:34:07.799-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:34:07.799-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:07.799-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:07.879-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:07.879-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:07.900-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:34:08.188-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:34:08.205-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:34:08.345-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:34:08.357-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:34:08.740-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:34:08.740-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:34:08.740-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:34:08.740-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:08.740-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:08.958-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:08.958-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:08.958-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:34:08.970-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:34:08.993-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:34:09.424-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:09.440-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:34:09.677-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:34:09.677-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:34:09.677-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:34:09.677-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:09.677-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:09.774-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:09.774-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:09.792-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:34:10.009-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:10.040-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:34:10.147-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:10.170-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:34:10.674-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:10.674-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:34:10.674-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:34:10.674-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:10.675-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:10.755-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:10.755-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:10.775-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:34:10.790-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:10.813-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:34:11.468-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:11.482-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
GenericSupervisor 2017-04-25T13:34:11.556-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:34:11.556-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:34:11.556-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:34:11.702-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:11.702-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:34:11.702-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:34:11.702-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:11.702-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:11.782-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:11.782-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:11.803-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:34:11.875-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:11.932-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
ToplogyChangeDetector 2017-04-25T13:34:11.955-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:34:12.250-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:12.264-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:34:12.477-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:12.477-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:34:12.477-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:34:12.477-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:12.477-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:12.540-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:12.541-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:12.557-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:34:12.665-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:12.685-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:34:12.994-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:34:13.010-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:34:13.174-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:13.174-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:34:13.174-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:34:13.174-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:13.174-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:13.296-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:13.296-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:13.330-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:34:13.434-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:34:13.461-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:34:13.798-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:34:13.816-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:34:14.001-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:34:14.001-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:34:14.001-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:34:14.001-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:14.001-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:14.083-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:14.083-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:14.096-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:34:14.150-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:34:14.161-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:34:14.650-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:34:14.650-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:34:14.650-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:34:14.650-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:14.650-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:34:14.654-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:34:14.654-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:34:14.654-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:14.654-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
PipelineRuntimeCtx 2017-04-25T13:34:14.654-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:34:14.654-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:34:14.655-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
GenericSupervisor 2017-04-25T13:34:14.658-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:34:14.658-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:14.655007804 -0500 CDT
GenericSupervisor 2017-04-25T13:34:14.661-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:14.661-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:34:14.666-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:14.666-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:34:14.666-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:14.666-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:34:14.667-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:34:14.668-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:34:14.668-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:34:14.668-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-492648327 has been stopped
GenericPipeline 2017-04-25T13:34:14.668-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-492648327 is stopped
PipelineManager 2017-04-25T13:34:14.668-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:34:14.668-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:04.498324766-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:34:14.668-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:14.668-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
GenericPipeline 2017-04-25T13:34:14.668-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
GenericPipeline 2017-04-25T13:34:14.668-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
GenericPipeline 2017-04-25T13:34:14.668-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-492648327
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:14.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:14.669-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:34:14.669-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:14.669-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:34:14.669-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:14.669-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:14.669392298 -0500 CDT
DcpNozzle 2017-04-25T13:34:14.669-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:14.669590874 -0500 CDT
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:34:14.669-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.669-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:14.670-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:34:14.674-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:14.674-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:14.674523949 -0500 CDT
DcpNozzle 2017-04-25T13:34:14.674-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:34:14.674-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:14.674-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:34:14 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:34:14 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:34:14 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:34:14 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:34:14 Stream Closed for vb 4 StreamEnd simulated
CapiNozzle 2017-04-25T13:34:14.678-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:34:14 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:34:14.713-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:14.713-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:14.713883871 -0500 CDT
DcpNozzle 2017-04-25T13:34:14.713-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:34:14.713-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:14.713-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:34:14.717-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:34:14.717-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:34:14.717-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:14.717-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:34:14.751-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:34:14.751-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:34:14.751-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:14.751-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:34:14.789-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:14.790-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:34:14.790-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:34:14.951-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:14.951-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:34:14.951-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:14.951-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:34:14.951-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:14.951-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:34:14.952-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:34:14.953-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:34:14.954-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
GenericSupervisor 2017-04-25T13:34:14.957-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:34:14.957-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
GenericSupervisor 2017-04-25T13:34:14.957-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:14.957-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:14.957-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:14.957-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:14.957-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:34:14.957-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
CheckpointManager 2017-04-25T13:34:14.957-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
CapiUtils 2017-04-25T13:34:15.058-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:15.058-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:15.058-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:34:15.059-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
StatisticsManager 2017-04-25T13:34:15.059-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:34:15.059-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
XDCRFactory 2017-04-25T13:34:15.060-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:34:15.060-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175 is constructed, start it
GenericPipeline 2017-04-25T13:34:15.061-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[30 129 72 75 137 142 18 105 107 168 29 99 143 55 128 23 38 42 126 135 76 78 119 56 60 165 70 83 102 150 43 47 57 20 79 50 65 89 86 136 1 8 12 94 145 27 59 158 0 17 26 81 139 146 44 115 87 95 48 41 71 91 118 163 93 96 156 164 166 22 53 154 52 73 120 14 97 98 68 109 6 19 77 140 15 32 110 138 160 144 104 133 151 36 39 58 2 82 124 134 5 112 28 61 167 63 132 162 170 31 123 125 7 122 159 127 35 103 130 169 116 149 51 62 161 16 88 101 92 45 113 34 80 147 11 40 121 4 100 33 157 85 90 37 46 74 25 131 13 106 108 10 84 117 141 148 3 66 69 21 49 9 111 24 54 64 153 155 67 114 152] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[227 174 310 196 210 300 173 279 188 205 309 185 225 231 257 270 334 182 315 219 261 267 215 229 311 232 286 298 203 220 283 319 194 183 186 295 324 340 222 299 325 333 335 317 234 242 297 198 305 338 277 287 331 290 271 274 296 306 211 175 236 268 292 221 263 312 327 197 214 209 191 269 339 244 201 199 252 192 253 254 291 330 249 281 204 303 307 200 241 280 179 239 189 190 248 202 213 246 285 171 320 328 217 262 264 302 322 243 216 272 275 282 284 172 193 326 245 207 240 304 233 313 318 180 177 218 266 308 329 247 288 258 228 195 208 238 293 332 187 336 230 235 265 184 276 176 256 321 301 314 250 260 289 337 224 323 178 212 237 273 316 255 223 226 181 206 294 251 259 278]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[228 258 187 195 208 238 293 332 336 230 235 184 265 276 176 256 321 260 301 314 250 289 337 212 224 323 178 237 273 316 223 255 206 226 181 294 251 259 278 227 174 310 196 210 300 173 279 188 205 309 185 225 231 334 257 270 315 182 215 219 261 267 229 298 311 232 286 194 203 220 283 319 183 186 295 324 340 222 299 325 333 335 234 317 198 242 297 287 305 338 277 331 290 211 271 274 296 306 175 236 268 292 214 221 263 312 327 197 209 191 269 339 201 244 252 199 192 253 254 291 330 204 249 281 241 303 307 200 239 280 179 189 190 248 202 213 246 285 171 320 328 217 262 264 302 322 243 193 216 272 275 282 284 172 326 245 207 240 304 180 233 313 318 177 218 266 288 308 329 247] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[45 113 34 80 121 147 11 40 4 100 33 157 74 85 90 37 46 25 108 131 13 106 10 84 69 117 141 148 3 66 21 49 9 64 111 24 54 152 153 155 67 114 30 129 137 72 75 107 142 18 105 143 168 29 99 55 128 42 23 38 119 126 135 76 78 56 60 165 57 70 83 102 150 43 47 20 79 89 50 65 12 86 136 1 8 94 145 26 27 59 158 0 17 146 81 139 44 115 87 95 48 41 71 163 91 118 93 96 154 156 164 166 22 53 120 52 73 98 14 97 68 77 109 6 19 110 140 15 32 138 160 144 58 104 133 151 36 39 2 82 124 134 5 112 167 28 61 162 63 132 125 170 31 123 159 7 122 127 35 169 103 130 116 149 51 62 101 161 16 88 92]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 active:true filter_expression: worker_batch_size:500 source_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi]
CheckpointManager 2017-04-25T13:34:15.061-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175...
AsyncListener 2017-04-25T13:34:15.088-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:15.088-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:15.088-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:15.088-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:15.088-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:15.089-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:34:15.089-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:34:15.089-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:34:15.089-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:15.089-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:34:15.089-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:34:15.089-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:34:15.089-05:00 [INFO] Checkpointing starts in 1800.851 sec
PipelineRuntimeCtx 2017-04-25T13:34:15.089-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:34:15.089-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:34:15.089-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:34:15.089-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:34:15.091-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:34:15.092-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:34:15.097-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:34:15.132-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:34:15.132-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
ToplogyChangeDetector 2017-04-25T13:34:15.135-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:34:15.135-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:34:15.149-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:15.149-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:34:15.155-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
AsyncListener 2017-04-25T13:34:15.178-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:34:15.178-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:34:15.178-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CapiUtils 2017-04-25T13:34:15.207-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:15.207-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:15.207-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:34:15.219-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:34:15.229-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:34:15.230-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:15.230-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:34:15.230-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:34:15.230-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:34:15.230-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:34:15.252-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:34:15.252-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:34:15.253-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:34:15.272-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:34:15.272-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:34:15.353-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:34:15.400-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:34:15.400-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:15.400-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:34:15.400-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:34:15.400-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:34:15.400-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:34:15.400-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175 is started----------
PipelineManager 2017-04-25T13:34:15.400-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:34:15.400-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:34:15.400-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:34:15.567-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:34:15.567-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:34:15.567-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:34:15.568-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:34:15 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:34:15.664-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:34:15.665-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:34:15.666-05:00 [ERROR] Received malformed response from tcp connection vb=100. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f4/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}

CapiNozzle 2017-04-25T13:34:15.666-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err Received malformed response from tcp connection vb=100. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f4/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}
.
CapiNozzle 2017-04-25T13:34:15.678-05:00 [INFO] Send batch count=169
CapiNozzle 2017-04-25T13:34:15.692-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:34:15.712-05:00 [INFO] Send batch count=500
2017/04/25 13:34:15 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 213 successful
CapiNozzle 2017-04-25T13:34:15.901-05:00 [ERROR] Received malformed response from tcp connection vb=100. err=malformed HTTP response "0_008_00049_128_16-3.3F\"}]\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", num_bytes=26, res_buf={0_008_00049_128_16-3.3F"}]}

CapiNozzle 2017-04-25T13:34:15.901-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err Received malformed response from tcp connection vb=100. err=malformed HTTP response "0_008_00049_128_16-3.3F\"}]\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", num_bytes=26, res_buf={0_008_00049_128_16-3.3F"}]}
.
CapiNozzle 2017-04-25T13:34:15.913-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:34:15 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:34:15 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 325 successful
StatisticsManager 2017-04-25T13:34:16.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=7, changes_left=315501
2017/04/25 13:34:16 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 237 successful
CapiNozzle 2017-04-25T13:34:16.156-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:16.186-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:34:16 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 259 successful
PipelineManager 2017-04-25T13:34:16.347-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:34:16 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 291 successful
CapiNozzle 2017-04-25T13:34:16.558-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:16.573-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:34:16 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 91 successful
CheckpointManager 2017-04-25T13:34:16.630-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
CheckpointManager 2017-04-25T13:34:16.630-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:34:16.641-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:34:16 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 195 successful
CapiNozzle 2017-04-25T13:34:16.665-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:34:16 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:34:16 UPR_STREAMREQ for vb 98 successful
DcpNozzle 2017-04-25T13:34:16.732-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:34:16.813-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:34:16.931-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:16.943-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:34:17.102-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:17.126-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:34:17.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=7, changes_left=315501
CapiNozzle 2017-04-25T13:34:17.336-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:17.336-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:34:17.336-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:34:17.336-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:34:17.336-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:34:17.344-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:34:17.344-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:34:17.346-05:00 [INFO] err_list=[{"time":"2017-04-25T13:34:17.34678528-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:34:17.346-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:34:17.428-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:17.428-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:17.429-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:17.429-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:34:17.429-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:34:17.429-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:17.429-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:17.461-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:34:17.495-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:17.495-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:17.538-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:34:17.792-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:17.830-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:34:17.985-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:18.002-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:34:18.102-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:34:18.102-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175 {"Errors": "[{\"time\":\"2017-04-25T13:34:17.34678528-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315501, "data_replicated": 755, "dcp_datach_length": 0, "dcp_dispatch_time": 12, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 377, "docs_filtered": 0, "docs_opt_repd": 3, "docs_processed": 7, "docs_received_from_dcp": 63785, "docs_rep_queue": 0, "docs_written": 3, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 3, "set_failed_cr_source": 377, "set_filtered": 0, "set_received_from_dcp": 63785, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:34:18.299-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:18.314-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:34:18.535-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:18.550-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:34:18.883-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:18.883-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:34:18.883-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:34:18.883-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:18.883-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:18.973-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:18.974-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:18.996-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:34:19.026-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:19.026-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:34:19.026-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:34:19.026-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:19.026-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:19.084-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:19.084-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:19.099-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:34:19.745-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:19.745-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:19.794-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:34:19.801-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:34:20.127-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:20.140-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:34:20.305-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:20.325-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:34:20.427-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:20.427-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:34:20.427-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:34:20.427-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:20.427-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:20.504-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:20.504-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:20.524-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:34:20.615-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:20.615-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:34:20.615-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:34:20.615-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:20.615-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:20.769-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:20.769-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:20.791-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:34:21.117-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:34:21.137-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
CapiNozzle 2017-04-25T13:34:21.400-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:21.428-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:34:21.605-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:34:21.623-05:00 [INFO] Retrying update docs for vb 306 for the 3th time
CapiNozzle 2017-04-25T13:34:21.796-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:21.839-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:34:22.026-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:34:22.026-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 306 after 3 retries
CapiNozzle 2017-04-25T13:34:22.026-05:00 [ERROR] Raise error condition batch update docs failed for vb 306 after 3 retries
GenericSupervisor 2017-04-25T13:34:22.026-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:22.026-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:22.169-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:22.169-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:22.200-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:34:22.232-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:22.232-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:34:22.232-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:34:22.232-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:22.232-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:22.352-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:22.352-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:22.369-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:34:22.683-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:34:22.700-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:34:22.894-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:34:22.907-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:34:23.335-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:34:23.350-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:34:23.715-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:34:23.727-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:34:23.863-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:34:23.863-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:34:23.863-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:34:23.863-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:23.863-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:23.944-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:23.944-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:23.957-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
CapiNozzle 2017-04-25T13:34:24.089-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:34:24.089-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:34:24.089-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:34:24.089-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:24.089-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:24.184-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:24.184-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:24.195-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
CapiNozzle 2017-04-25T13:34:24.341-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:34:24.352-05:00 [INFO] Retrying update docs for vb 208 for the 2th time
CapiNozzle 2017-04-25T13:34:24.615-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:34:24.626-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
CapiNozzle 2017-04-25T13:34:24.805-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:34:24.816-05:00 [INFO] Retrying update docs for vb 208 for the 3th time
CapiNozzle 2017-04-25T13:34:24.951-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:34:24.961-05:00 [INFO] Retrying update docs for vb 105 for the 3th time
GenericSupervisor 2017-04-25T13:34:25.089-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:34:25.090-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:34:25.090-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:34:25.187-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:34:25.187-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 208 after 3 retries
CapiNozzle 2017-04-25T13:34:25.187-05:00 [ERROR] Raise error condition batch update docs failed for vb 208 after 3 retries
GenericSupervisor 2017-04-25T13:34:25.188-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:25.188-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:25.274-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:25.274-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:25.285-05:00 [INFO] Retrying update docs for vb 209 for the 1th time
ToplogyChangeDetector 2017-04-25T13:34:25.331-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:34:25.618-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:34:25.618-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 105 after 3 retries
CapiNozzle 2017-04-25T13:34:25.618-05:00 [ERROR] Raise error condition batch update docs failed for vb 105 after 3 retries
GenericSupervisor 2017-04-25T13:34:25.619-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:25.619-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:25.756-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:25.756-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:25.767-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:34:26.057-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:34:26.068-05:00 [INFO] Retrying update docs for vb 209 for the 2th time
CapiNozzle 2017-04-25T13:34:26.268-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:34:26.279-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:34:26.476-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:34:26.488-05:00 [INFO] Retrying update docs for vb 209 for the 3th time
CapiNozzle 2017-04-25T13:34:26.774-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:34:26.785-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:34:27.025-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:34:27.025-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 209 after 3 retries
CapiNozzle 2017-04-25T13:34:27.025-05:00 [ERROR] Raise error condition batch update docs failed for vb 209 after 3 retries
GenericSupervisor 2017-04-25T13:34:27.025-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:27.025-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:27.096-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:27.096-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:27.107-05:00 [INFO] Retrying update docs for vb 307 for the 1th time
CapiNozzle 2017-04-25T13:34:27.242-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:34:27.243-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:34:27.243-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:34:27.243-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:27.243-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:27.345-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:27.346-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:34:27.347-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:34:27.347-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:34:27.347-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:27.347-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
PipelineRuntimeCtx 2017-04-25T13:34:27.347-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:34:27.347-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:34:27.347-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:34:27.347-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:34:27.347-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:27.347-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:34:27.347-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:27.347512993 -0500 CDT
GenericSupervisor 2017-04-25T13:34:27.347-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:27.347-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:34:27.347-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:27.347-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:34:27.347-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:27.347-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:34:27.347-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-952024175 has been stopped
GenericPipeline 2017-04-25T13:34:27.348-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-952024175 is stopped
PipelineManager 2017-04-25T13:34:27.348-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:34:27.348-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:17.34678528-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:34:27.348-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:27.348-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
GenericPipeline 2017-04-25T13:34:27.348-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
GenericPipeline 2017-04-25T13:34:27.348-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
GenericPipeline 2017-04-25T13:34:27.348-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-952024175
AsyncListener 2017-04-25T13:34:27.348-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:27.348-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:27.348-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:27.348-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:27.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:34:27.349-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:27.349-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:34:27.349-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:34:27.349-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:27.349-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:34:27.349-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:34:27.349-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:27.349-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:34:27.349-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:27.350-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:34:27.350-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:27.350-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:27.350-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:34:27.350-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:27.349838794 -0500 CDT
DcpNozzle 2017-04-25T13:34:27.350-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:34:27.350-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:27.350146139 -0500 CDT
CapiNozzle 2017-04-25T13:34:27.350-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:34:27.351-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:34:27.351-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:34:27.351-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:27.351-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:34:27.352-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:27.352-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:34:27 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:34:27.356-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:27.356-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:27.356044546 -0500 CDT
DcpNozzle 2017-04-25T13:34:27.356-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:27.356-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:34:27.356-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:27.356173665 -0500 CDT
GenericPipeline 2017-04-25T13:34:27.356-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:27.356-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:34:27.356-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:34:27.356-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:27.356-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:34:27 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:34:27 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:34:27.370-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:34:27.428-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:34:27.428-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:34:27.428-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:27.429-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:34:27.507-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:27.507-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:34:27.508-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:34:27.563-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:27.563-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:34:27.563-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:27.563-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:34:27.563-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:27.564-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:34:27.564-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:34:27.564-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:34:27.564-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
GenericSupervisor 2017-04-25T13:34:27.567-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:34:27.567-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
GenericSupervisor 2017-04-25T13:34:27.567-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:27.567-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:27.567-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:27.567-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:27.567-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:34:27.567-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
CheckpointManager 2017-04-25T13:34:27.567-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
CapiUtils 2017-04-25T13:34:27.599-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:27.599-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:27.599-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:34:27.600-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
StatisticsManager 2017-04-25T13:34:27.600-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:34:27.600-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
XDCRFactory 2017-04-25T13:34:27.600-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:34:27.601-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052 is constructed, start it
GenericPipeline 2017-04-25T13:34:27.602-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[271 294 211 219 324 216 256 189 237 255 259 225 230 289 261 263 332 176 179 269 319 173 203 280 191 274 303 333 221 267 315 323 318 326 336 245 212 250 205 195 202 209 210 293 186 217 262 240 185 201 208 248 192 246 310 328 298 265 181 190 229 197 242 272 283 244 270 276 254 222 232 275 175 238 174 200 214 233 299 317 330 314 277 290 227 285 206 273 180 204 241 266 322 331 177 198 235 247 281 284 311 264 171 215 231 321 296 178 220 218 223 287 252 187 228 253 316 334 257 288 305 236 320 279 306 194 295 312 268 340 239 207 249 301 182 234 308 183 251 258 260 286 300 313 339 172 327 291 337 193 184 307 188 196 213 282 292 302 199 226 325 329 243 278 297 304 309 335 338 224] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[91 145 54 78 102 117 121 146 164 48 77 111 19 59 126 22 80 158 56 71 106 125 25 74 167 101 127 89 119 135 168 9 83 26 45 70 75 108 18 29 93 65 120 6 31 97 3 130 39 105 5 20 13 53 41 113 34 40 21 51 110 4 17 1 88 122 160 27 94 118 57 95 76 86 37 68 116 149 66 96 87 143 150 11 43 46 163 139 151 8 132 133 153 156 157 166 47 100 38 154 128 14 98 107 124 33 60 92 99 123 2 81 109 90 32 61 104 15 16 36 136 30 52 144 161 165 170 0 114 62 159 134 138 50 137 23 84 155 69 73 12 42 35 147 7 169 44 63 115 148 142 152 58 129 24 55 131 140 28 64 82 162 72 85 10 67 49 112 103 141 79]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[279 306 194 295 312 268 340 239 207 249 301 182 234 308 183 251 258 260 286 300 313 339 172 327 291 337 193 184 307 188 196 213 282 292 302 199 226 325 329 243 278 297 304 309 335 338 224 271 294 211 219 324 216 256 189 237 255 259 225 230 289 261 263 332 176 179 269 319 173 203 280 191 274 303 333 221 267 315 323 318 326 336 245 212 250 205 195 202 209 210 293 186 217 262 240 185 201 208 248 192 246 310 328 298 265 181 190 229 197 242 272 283 244 270 276 254 222 232 275 175 238 174 200 214 233 299 317 330 314 277 290 227 285 206 273 180 204 241 266 322 331 177 198 235 247 281 284 311 264 171 215 231 321 296 178 220 218 223 287 252 187 228 253 316 334 257 288 305 236 320] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[134 138 50 137 23 84 155 69 73 12 42 35 147 7 169 44 63 115 148 142 152 58 129 24 55 131 140 28 64 82 162 72 85 10 67 49 112 103 141 79 91 145 54 78 102 117 121 146 164 48 77 111 19 59 126 22 80 158 56 71 106 125 25 74 167 101 127 89 119 135 168 9 83 26 45 70 75 108 18 29 93 65 120 6 31 97 3 130 39 105 5 20 13 53 41 113 34 40 21 51 110 4 17 1 88 122 160 27 94 118 57 95 76 86 37 68 116 149 66 96 87 143 150 11 43 46 163 139 151 8 132 133 153 156 157 166 47 100 38 154 128 14 98 107 124 33 60 92 99 123 2 81 109 90 32 61 104 15 16 36 136 30 52 144 161 165 170 0 114 62 159]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi doc_batch_size_kb:2048 failure_restart_interval:10 stats_interval:1000 filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:34:27.603-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052...
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:34:27.611-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:34:27.611-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:27.611-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:34:27.611-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:34:27.611-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:34:27.611-05:00 [INFO] Checkpointing starts in 1800.275 sec
PipelineRuntimeCtx 2017-04-25T13:34:27.611-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:34:27.612-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:34:27.612-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:34:27.612-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:34:27.614-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:34:27.614-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:34:27.614-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:34:27.615-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:34:27.615-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:34:27.639-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:34:27.639-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:34:27.639-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:34:27.670-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:27.670-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:34:27.676-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:34:27.683-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:27.683-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:27.683-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:34:27.688-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:34:27.703-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:34:27.706-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:34:27.706-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:27.706-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:34:27.706-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:34:27.706-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:34:27.707-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:34:27.731-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:34:27.731-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:34:27.742-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:34:27.753-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:34:27.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:34:27.854-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:34:27.904-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:34:27.904-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:27.904-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:34:27.904-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:34:27.904-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:34:27.904-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:34:27.905-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052 is started----------
PipelineManager 2017-04-25T13:34:27.905-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:34:27.905-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:34:27.905-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:34:28.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:34:28.931-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:34:28.932-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:34:28.932-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:34:28.932-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:34:29 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:34:29.054-05:00 [INFO] Send batch count=202
CapiNozzle 2017-04-25T13:34:29.055-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:34:29.061-05:00 [INFO] Send batch count=182
CapiNozzle 2017-04-25T13:34:29.118-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:29.118-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:34:29 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:34:29.182-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:34:29 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 21 successful
CapiNozzle 2017-04-25T13:34:29.421-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:29.436-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:34:29 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 131 successful
CapiNozzle 2017-04-25T13:34:29.558-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:29.584-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:34:29 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 239 successful
StatisticsManager 2017-04-25T13:34:29.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:34:29 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 251 successful
CapiNozzle 2017-04-25T13:34:29.782-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:29.800-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:34:29 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:34:29 UPR_STREAMREQ for vb 269 successful
CapiNozzle 2017-04-25T13:34:29.928-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:29.955-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:34:30 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 180 successful
CapiNozzle 2017-04-25T13:34:30.087-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:30.104-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
2017/04/25 13:34:30 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 292 successful
CheckpointManager 2017-04-25T13:34:30.276-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
CheckpointManager 2017-04-25T13:34:30.276-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:34:30 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:34:30 UPR_STREAMREQ for vb 98 successful
CapiNozzle 2017-04-25T13:34:30.356-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:30.356-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:34:30.356-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:34:30.356-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:34:30.356-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:34:30.368-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:34:30.369-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:34:30.371-05:00 [INFO] err_list=[{"time":"2017-04-25T13:34:30.371914011-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:34:30.372-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
DcpNozzle 2017-04-25T13:34:30.406-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:34:30.425-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:34:30.454-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:30.454-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:30.498-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:34:30.511-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:30.511-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:34:30.511-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:34:30.511-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:30.511-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:30.584-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:30.584-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:30.612-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
StatisticsManager 2017-04-25T13:34:30.633-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:34:30.633-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052 {"Errors": "[{\"time\":\"2017-04-25T13:34:30.371914011-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 156, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 21328, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 156, "set_filtered": 0, "set_received_from_dcp": 21328, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:34:31.465-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:31.466-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:31.489-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:34:31.503-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
PipelineManager 2017-04-25T13:34:31.588-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:30.371914011-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:34:31.828-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:31.870-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:34:31.995-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:32.013-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:34:32.122-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:32.122-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:34:32.122-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:34:32.122-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:32.122-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:32.186-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:32.186-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:32.219-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:34:32.279-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:32.279-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:34:32.279-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:34:32.279-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:32.280-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:32.374-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:32.374-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:32.388-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:34:32.639-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:32.656-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:34:32.840-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:32.862-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:34:32.990-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:33.005-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:34:33.222-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:33.235-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:34:34.123-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:34.123-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:34:34.124-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:34:34.124-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:34.124-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:34.235-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:34.235-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:34.266-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:34:34.443-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:34.443-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:34:34.443-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:34:34.443-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:34.443-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:34.516-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:34.517-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:34.535-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:34:34.555-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:34.580-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:34:35.012-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:35.030-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:34:35.257-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:35.270-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:34:35.477-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:35.493-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:34:35.742-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:35.742-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:34:35.742-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:34:35.742-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:35.742-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:35.840-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:35.840-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:35.862-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:34:35.951-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:35.951-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:34:35.951-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:34:35.951-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:35.951-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:36.081-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:36.081-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:36.095-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:34:36.224-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:36.245-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:34:36.438-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:36.456-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:34:36.650-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:36.675-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:34:36.936-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:36.956-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:34:37.077-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:37.077-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:34:37.077-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:34:37.077-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:37.077-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:37.164-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:37.165-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:37.182-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:34:37.513-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:37.513-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:34:37.513-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:34:37.513-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:37.513-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:37.595-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:37.595-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:37.605-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
GenericSupervisor 2017-04-25T13:34:37.612-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:34:37.612-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:34:37.612-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:34:37.752-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:34:37.762-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
ToplogyChangeDetector 2017-04-25T13:34:37.874-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:34:38.227-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:38.238-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:34:38.386-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:34:38.397-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:34:38.719-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:38.730-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:34:38.903-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:34:38.903-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:34:38.903-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:34:38.903-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:38.903-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:38.975-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:38.975-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:38.986-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:34:39.083-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:39.083-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:34:39.083-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:34:39.083-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:39.083-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:39.177-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:39.177-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:39.188-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:34:39.225-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:34:39.237-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:34:39.706-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:34:39.716-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
CapiNozzle 2017-04-25T13:34:40.362-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
PipelineManager 2017-04-25T13:34:40.372-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:34:40.372-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:34:40.372-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:40.372-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
PipelineRuntimeCtx 2017-04-25T13:34:40.372-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:34:40.372-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:40.372-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:34:40.372-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:40.372449345 -0500 CDT
GenericSupervisor 2017-04-25T13:34:40.372-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:40.372-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:34:40.372-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:40.372-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:34:40.372-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:40.372-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:34:40.372-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:34:40.372-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:34:40.372-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:34:40.372-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-564158052 has been stopped
GenericPipeline 2017-04-25T13:34:40.372-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-564158052 is stopped
PipelineManager 2017-04-25T13:34:40.373-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:34:40.373-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:30.371914011-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:34:40.373-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-564158052
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:34:40.373-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:34:40.373-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:40.373-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:40.373-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:34:40.373-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:34:40.373-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:34:40.373-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:34:40.373-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:34:40.374-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:40.374-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:40.373912665 -0500 CDT
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:40.374-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:40.374245797 -0500 CDT
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:34:40.374-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:34:40.374-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.375-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:40.375-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:40.375-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:34:40.375-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:40.375-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:34:40.375-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:40.376-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:40.375986098 -0500 CDT
DcpNozzle 2017-04-25T13:34:40.376-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:34:40.376-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:40.376-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:34:40.376-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:40.376-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:40.37675254 -0500 CDT
DcpNozzle 2017-04-25T13:34:40.376-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:34:40.376-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:40.376-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:34:40 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:34:40 Stream Closed for vb 1 StreamEnd simulated
CapiNozzle 2017-04-25T13:34:40.377-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
2017/04/25 13:34:40 Stream Closed for vb 171 StreamEnd simulated
CapiNozzle 2017-04-25T13:34:40.438-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:34:40.438-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:34:40.438-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:40.438-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:34:40.528-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:40.528-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:34:40.528-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:34:40.617-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:40.617-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:34:40.618-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:40.618-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:34:40.618-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:40.618-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:34:40.618-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:34:40.618-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:34:40.618-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
GenericSupervisor 2017-04-25T13:34:40.620-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:34:40.620-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
GenericSupervisor 2017-04-25T13:34:40.620-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:40.620-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:40.620-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:40.620-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:40.620-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:34:40.620-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
CheckpointManager 2017-04-25T13:34:40.620-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
CapiUtils 2017-04-25T13:34:40.648-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:40.648-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:40.648-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:34:40.649-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
StatisticsManager 2017-04-25T13:34:40.649-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:34:40.649-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
XDCRFactory 2017-04-25T13:34:40.650-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:34:40.650-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984 is constructed, start it
GenericPipeline 2017-04-25T13:34:40.651-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[110 119 70 14 27 40 49 160 3 66 112 164 47 38 128 157 162 34 29 104 23 2 107 148 81 92 89 45 69 136 21 86 55 126 143 82 116 87 156 11 115 123 72 43 26 91 24 35 101 127 144 158 1 56 105 124 169 48 118 153 13 32 59 18 98 108 132 133 54 65 85 142 152 46 36 53 6 61 4 141 39 41 67 51 111 155 165 37 154 63 80 75 103 121 130 33 57 95 139 12 106 117 129 17 90 102 138 28 166 20 64 60 147 73 97 114 0 135 42 83 19 94 77 44 31 58 131 159 8 122 140 88 120 170 16 167 25 50 68 76 7 93 99 109 78 161 30 79 96 113 163 74 52 149 9 100 10 22 125 137 168 5 134 145 146 71 62 84 150 151 15] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[197 231 293 295 302 178 213 287 303 328 307 174 198 339 311 324 195 210 244 260 281 315 205 227 241 253 211 334 186 188 289 314 196 275 272 318 175 264 265 292 209 225 270 335 336 217 269 202 206 297 313 278 199 224 288 180 258 261 304 216 221 319 333 187 204 266 190 182 219 222 243 301 245 267 193 172 181 248 299 233 300 207 236 229 251 325 230 268 283 332 338 176 189 194 200 331 279 286 291 326 340 177 203 208 237 317 235 259 171 192 282 273 322 185 212 252 305 316 249 274 290 306 309 310 323 280 285 320 239 294 330 242 298 321 327 234 262 263 312 223 183 238 247 255 296 271 218 220 250 284 308 337 184 256 173 254 277 201 246 276 214 232 240 215 228 179 329 191 226 257]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[113 163 74 79 96 52 9 149 10 100 137 168 5 22 125 146 71 134 145 150 151 15 62 84 70 110 119 40 49 160 3 14 27 164 47 66 112 157 162 34 38 128 23 29 104 2 81 107 148 89 92 136 21 45 69 55 86 82 126 143 116 11 87 156 72 115 123 43 24 26 91 127 144 158 1 35 101 124 169 48 56 105 13 118 153 18 32 59 132 133 54 98 108 65 152 46 85 142 6 36 53 4 61 39 141 41 67 155 165 37 51 111 63 154 75 80 130 33 103 121 139 12 57 95 129 17 106 117 138 28 90 102 20 166 60 64 147 114 0 73 97 135 19 42 83 77 94 44 131 159 8 31 58 122 140 170 16 88 120 25 167 50 7 68 76 109 78 93 99 30 161] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[184 256 277 173 254 201 276 246 214 232 240 179 215 228 329 191 226 257 197 231 293 295 302 178 213 287 303 328 174 307 198 339 311 324 244 260 281 315 195 210 227 241 253 205 211 334 186 188 289 314 196 275 272 318 175 264 265 292 270 335 336 209 225 217 269 202 206 278 297 313 199 224 288 180 258 261 304 216 221 319 333 187 204 266 190 222 243 301 182 219 245 267 172 193 248 299 181 207 233 300 236 229 251 325 230 268 283 332 338 176 189 194 200 331 286 291 326 340 279 203 177 317 208 237 235 259 192 282 171 273 322 185 305 316 212 252 290 306 309 310 323 249 274 280 285 320 294 330 239 242 298 321 327 263 312 223 234 262 183 238 247 255 271 296 220 250 284 308 337 218]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info active:true checkpoint_interval:1800 target_nozzle_per_node:2 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500]
CheckpointManager 2017-04-25T13:34:40.651-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984...
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:40.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:34:40.668-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:34:40.668-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:34:40.668-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:40.668-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:34:40.668-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:34:40.668-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:34:40.668-05:00 [INFO] Checkpointing starts in 1804.612 sec
PipelineRuntimeCtx 2017-04-25T13:34:40.668-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:34:40.668-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:34:40.668-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:34:40.668-05:00 [INFO] checkpointing rountine started
ToplogyChangeDetector 2017-04-25T13:34:40.675-05:00 [INFO] validateSourceTopology...
PipelineRuntimeCtx 2017-04-25T13:34:40.681-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:34:40.681-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:34:40.685-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:34:40.687-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:40.687-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:40.687-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:34:40.692-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:34:40.757-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:34:40.757-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:34:40.757-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:34:40.776-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:34:40.777-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:34:40.777-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:34:40.823-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:34:40.825-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:34:40.825-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:40.825-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:34:40.825-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:34:40.825-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:34:40.825-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:34:40.862-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:34:40.862-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:34:40.862-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:34:40.881-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:40.881-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:41.034-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:34:41.034-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:41.034-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:34:41.056-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:34:41.056-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:34:41.056-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:34:41.057-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984 is started----------
PipelineManager 2017-04-25T13:34:41.057-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:34:41.057-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:34:41.057-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:34:41.057-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:34:41.057-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:34:41.058-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:34:41.058-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:34:41 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:34:41.254-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:34:41.254-05:00 [INFO] Send batch count=45
2017/04/25 13:34:41 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 304 successful
CapiNozzle 2017-04-25T13:34:41.305-05:00 [INFO] Send batch count=146
CapiNozzle 2017-04-25T13:34:41.340-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:41.341-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:34:41 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 305 successful
CapiNozzle 2017-04-25T13:34:41.373-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:34:41 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 330 successful
StatisticsManager 2017-04-25T13:34:41.747-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:34:41 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 239 successful
CapiNozzle 2017-04-25T13:34:41.801-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:41.830-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:34:41 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:34:41 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 73 successful
CapiNozzle 2017-04-25T13:34:42.058-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:34:42 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 272 successful
CapiNozzle 2017-04-25T13:34:42.082-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:42.091-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:34:42.104-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
2017/04/25 13:34:42 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 85 successful
CheckpointManager 2017-04-25T13:34:42.146-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
CheckpointManager 2017-04-25T13:34:42.146-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:34:42.173-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:34:42 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 280 successful
CapiNozzle 2017-04-25T13:34:42.191-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:34:42 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:34:42 UPR_STREAMREQ for vb 95 successful
DcpNozzle 2017-04-25T13:34:42.267-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:34:42.325-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
DcpNozzle 2017-04-25T13:34:42.328-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:34:42.337-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:42.337-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:34:42.337-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:34:42.337-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
ReplicationManager 2017-04-25T13:34:42.337-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
PipelineManager 2017-04-25T13:34:42.348-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:34:42.348-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:42.348-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
PipelineManager 2017-04-25T13:34:42.350-05:00 [INFO] err_list=[{"time":"2017-04-25T13:34:42.350949833-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]
PipelineManager 2017-04-25T13:34:42.351-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:34:42.469-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:42.474-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:42.486-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:34:42.574-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:34:42.574-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:34:42.574-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:34:42.574-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:42.574-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:42.634-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:42.634-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:42.670-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
StatisticsManager 2017-04-25T13:34:42.685-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:34:42.685-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984 {"Errors": "[{\"time\":\"2017-04-25T13:34:42.350949833-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 22, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 71, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 21261, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 71, "set_filtered": 0, "set_received_from_dcp": 21261, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:34:42.959-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:42.973-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:34:43.259-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:43.290-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:34:43.555-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:43.570-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:34:44.008-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:44.027-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:34:44.217-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:44.217-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:34:44.217-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:34:44.217-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:44.217-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:44.336-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:44.336-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:44.351-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:34:44.546-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:44.546-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:34:44.546-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:34:44.546-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:44.546-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:44.628-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:44.628-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:44.651-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:34:46.139-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:46.139-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:46.191-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:34:46.214-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:34:46.436-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:42.350949833-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]}
]
CapiNozzle 2017-04-25T13:34:46.536-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:46.557-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
ReplicationManager 2017-04-25T13:34:46.640-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:34:46.821-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:46.846-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:34:47.039-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:47.039-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:34:47.039-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:34:47.039-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:47.039-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:47.102-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:47.102-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:47.131-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:34:47.495-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:47.495-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:34:47.495-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:34:47.495-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:47.495-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:47.634-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:47.648-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:47.685-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:34:47.968-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:47.986-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:34:48.141-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:48.164-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:34:48.543-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:48.565-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:34:48.701-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:48.715-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:34:49.198-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:49.198-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:34:49.198-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:34:49.198-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:49.198-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:49.304-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:49.304-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:49.317-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:34:49.363-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:34:49.363-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:34:49.363-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:34:49.363-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:49.363-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:49.427-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:49.427-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:49.449-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:34:49.678-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:49.711-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:34:50.423-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:50.440-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
GenericSupervisor 2017-04-25T13:34:50.672-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:34:50.672-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:34:50.672-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:34:50.699-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:50.723-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
ToplogyChangeDetector 2017-04-25T13:34:50.814-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:34:51.047-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:51.058-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:34:51.209-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:51.209-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:34:51.209-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:34:51.209-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:51.209-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:51.292-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:51.293-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:51.303-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:34:51.451-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:34:51.451-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:34:51.451-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:34:51.451-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:51.451-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:51.542-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:51.543-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:51.553-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:34:51.923-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:34:51.934-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:34:52.235-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:34:52.246-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
PipelineManager 2017-04-25T13:34:52.351-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries 
PipelineManager 2017-04-25T13:34:52.351-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:34:52.351-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:52.351-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
PipelineRuntimeCtx 2017-04-25T13:34:52.351-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:34:52.351-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:52.351-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:34:52.351-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:52.351479945 -0500 CDT
GenericSupervisor 2017-04-25T13:34:52.351-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:52.351-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:34:52.351-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:52.351-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:34:52.351-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:34:52.351-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:34:52.351-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:34:52.351-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:34:52.351-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:34:52.351-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-618251984 has been stopped
GenericPipeline 2017-04-25T13:34:52.352-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-618251984 is stopped
PipelineManager 2017-04-25T13:34:52.352-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:34:52.352-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:42.350949833-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:34:52.352-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:34:52.352-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
GenericPipeline 2017-04-25T13:34:52.352-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
GenericPipeline 2017-04-25T13:34:52.352-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
GenericPipeline 2017-04-25T13:34:52.352-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-618251984
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:34:52.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:52.353-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:34:52.353-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:52.353-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:34:52.353-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:52.353-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:34:52.353-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:34:52.353-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:34:52.353-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:52.353-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:34:52.353-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:34:52.353-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:34:52.353-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:52.353389279 -0500 CDT
CapiNozzle 2017-04-25T13:34:52.354-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:34:52.354-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:34:52.354-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.354-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:34:52.354-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:52.354-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:34:52.357-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:52.357-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:52.357691644 -0500 CDT
DcpNozzle 2017-04-25T13:34:52.357-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:34:52.357-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:52.357-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:34:52 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:34:52 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:34:52 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:34:52 Stream Closed for vb 174 StreamEnd simulated
CapiNozzle 2017-04-25T13:34:52.358-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:52.353657567 -0500 CDT
CapiNozzle 2017-04-25T13:34:52.358-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:34:52.359-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:34:52.359-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.359-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:34:52.359-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:34:52.359-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.359-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:34:52.359-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:52.359-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:34:52.360-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:34:52.360-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:34:52.360593195 -0500 CDT
DcpNozzle 2017-04-25T13:34:52.360-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:34:52.360-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:34:52.360-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
2017/04/25 13:34:52 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:34:52 Stream Closed for vb 1 StreamEnd simulated
XDCRFactory 2017-04-25T13:34:52.474-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:52.474-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:34:52.474-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:34:52.537-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:34:52.537-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:34:52.537-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:52.537-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:34:52.537-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:34:52.537-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:34:52.537-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:34:52.538-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:34:52.538-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
GenericSupervisor 2017-04-25T13:34:52.539-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:34:52.539-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
GenericSupervisor 2017-04-25T13:34:52.539-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:52.539-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:52.539-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:52.539-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:34:52.539-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:34:52.539-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
CheckpointManager 2017-04-25T13:34:52.539-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
CapiUtils 2017-04-25T13:34:52.558-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:52.558-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:52.558-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:34:52.559-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
StatisticsManager 2017-04-25T13:34:52.559-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:34:52.559-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
XDCRFactory 2017-04-25T13:34:52.559-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:34:52.559-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854 is constructed, start it
GenericPipeline 2017-04-25T13:34:52.560-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[40 107 24 140 104 29 46 53 121 2 27 76 12 112 168 98 48 79 11 36 25 152 126 52 85 31 94 162 129 103 148 75 41 136 159 91 50 73 86 139 141 60 15 33 61 95 99 119 10 57 142 14 137 63 133 155 156 166 54 45 164 0 74 123 93 78 90 143 49 170 30 66 80 22 47 92 110 161 13 109 116 134 51 88 62 39 20 144 127 117 64 87 102 55 125 83 67 132 167 18 84 114 130 23 135 37 7 43 96 124 4 65 68 38 113 122 82 89 32 97 100 71 17 34 149 8 106 154 81 42 59 69 21 26 101 108 118 147 19 6 77 163 3 58 146 158 5 151 157 72 150 131 120 165 44 9 35 56 115 128 1 111 16 160 138 70 105 153 28 169 145] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[237 265 307 190 211 204 216 271 213 229 256 336 268 324 325 246 298 194 203 275 279 310 192 201 223 231 319 322 182 257 270 175 291 304 172 234 260 221 276 326 339 331 202 299 311 205 188 308 330 196 184 239 317 292 300 183 236 318 274 277 321 249 313 337 171 269 210 294 297 312 186 225 266 284 198 199 241 206 228 259 309 248 273 287 263 306 340 327 217 218 303 176 193 240 286 174 208 293 222 235 209 230 247 250 295 302 255 173 187 220 262 280 281 185 243 254 285 290 283 200 251 264 333 178 195 335 226 245 207 242 282 197 215 278 305 180 232 316 219 227 272 328 332 252 329 191 261 288 189 224 320 177 181 214 267 338 314 323 179 244 296 315 334 253 301 212 233 258 289 238]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[72 151 157 131 150 120 44 165 128 1 9 35 56 115 16 111 138 160 28 70 105 153 145 169 24 40 107 104 140 2 29 46 53 121 12 27 76 112 98 168 11 48 79 25 36 126 152 31 52 85 94 129 162 75 103 148 41 91 136 159 50 60 73 86 139 141 99 119 10 15 33 61 95 14 57 142 137 166 54 63 133 155 156 0 45 164 74 123 93 49 78 90 143 30 170 22 66 80 47 13 92 110 161 109 51 116 134 62 88 20 39 127 144 117 64 55 87 102 83 125 67 18 132 167 23 84 114 130 37 135 4 7 43 96 124 38 65 68 82 113 122 32 89 71 97 100 8 17 34 149 81 106 154 21 42 59 69 147 19 26 101 108 118 3 6 77 163 5 58 146 158] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[252 329 191 261 288 189 224 320 267 338 177 181 214 314 323 315 334 179 244 296 253 301 233 258 289 212 238 307 237 265 190 211 204 216 271 213 229 256 336 268 324 325 246 298 275 279 310 194 203 192 201 223 231 319 322 182 257 270 304 175 291 172 234 260 221 276 326 331 339 202 205 299 311 330 188 308 196 184 239 317 292 300 183 236 318 277 321 274 249 313 337 171 269 210 294 297 312 225 266 284 186 198 199 241 259 309 206 228 248 273 287 263 306 340 327 217 218 303 240 286 176 193 174 208 222 293 235 230 247 250 209 255 295 302 262 280 281 173 187 220 185 243 254 283 285 290 264 333 200 251 335 178 195 226 245 282 207 242 197 215 278 305 232 316 180 227 272 328 219 332]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[optimistic_replication_threshold:256 stats_interval:1000 active:true doc_batch_size_kb:2048 failure_restart_interval:10 worker_batch_size:500 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: checkpoint_interval:1800]
CheckpointManager 2017-04-25T13:34:52.560-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854...
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:34:52.566-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:34:52.566-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:34:52.566-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:34:52.566-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:34:52.566-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:34:52.566-05:00 [INFO] Checkpointing starts in 1803.225 sec
PipelineRuntimeCtx 2017-04-25T13:34:52.566-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:34:52.566-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:34:52.566-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:34:52.566-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:34:52.567-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:34:52.568-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:34:52.567-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:34:52.568-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:34:52.568-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:34:52.592-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:34:52.593-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:34:52.593-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:34:52.614-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:34:52.614-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:34:52.617-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:34:52.617-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:34:52.617-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:34:52.621-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:34:52.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:34:52.636-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:34:52.654-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:34:52.654-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:52.654-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:34:52.654-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:34:52.654-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:34:52.656-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:34:52.685-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:34:52.686-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:34:52.686-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:34:52.708-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:34:52.708-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:34:52.787-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:34:52.828-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:34:52.828-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:34:52.828-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:34:52.828-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:34:52.828-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:34:52.828-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:34:52.828-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854 is started----------
PipelineManager 2017-04-25T13:34:52.828-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:34:52.828-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:34:52.828-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
AsyncListener 2017-04-25T13:34:52.829-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:34:52.829-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
StatisticsManager 2017-04-25T13:34:53.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:34:54.664-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:34:55.467-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:34:55.468-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:34:55.468-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:34:55.468-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:34:55 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:34:55.573-05:00 [INFO] Send batch count=61
2017/04/25 13:34:55 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 4 successful
CapiNozzle 2017-04-25T13:34:55.602-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:55.602-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:55.620-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:34:55.627-05:00 [INFO] Send batch count=125
StatisticsManager 2017-04-25T13:34:55.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:34:55 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 8 successful
CapiNozzle 2017-04-25T13:34:55.728-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:55.728-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:55.752-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
2017/04/25 13:34:55 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 316 successful
CapiNozzle 2017-04-25T13:34:55.786-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:55.817-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:34:55 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:34:55 UPR_STREAMREQ for vb 135 successful
CapiNozzle 2017-04-25T13:34:56.015-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:56.038-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
2017/04/25 13:34:56 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 40 successful
CapiNozzle 2017-04-25T13:34:56.114-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:56.140-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
2017/04/25 13:34:56 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 263 successful
CapiNozzle 2017-04-25T13:34:56.247-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:56.260-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
2017/04/25 13:34:56 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 273 successful
CapiNozzle 2017-04-25T13:34:56.351-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:34:56.351-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:34:56.351-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:34:56.351-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
ReplicationManager 2017-04-25T13:34:56.351-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
PipelineManager 2017-04-25T13:34:56.363-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:34:56.364-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:34:56.366-05:00 [INFO] err_list=[{"time":"2017-04-25T13:34:56.366966613-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:34:56.367-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:34:56 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 78 successful
CapiNozzle 2017-04-25T13:34:56.441-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:56.441-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:34:56 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 281 successful
CapiNozzle 2017-04-25T13:34:56.461-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:56.461-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:34:56.461-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:34:56.461-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:56.461-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:56.476-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:34:56 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 83 successful
CapiNozzle 2017-04-25T13:34:56.522-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:56.522-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CheckpointManager 2017-04-25T13:34:56.540-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
CheckpointManager 2017-04-25T13:34:56.540-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:34:56.551-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:34:56 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 291 successful
StatisticsManager 2017-04-25T13:34:56.592-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:34:56.592-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854 {"Errors": "[{\"time\":\"2017-04-25T13:34:56.366966613-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 6, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 191, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 191, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]", "Status": "Pending"}

2017/04/25 13:34:56 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:34:56 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:34:56.630-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:34:56.656-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:34:56.918-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:56.935-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:34:57.164-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:57.200-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:34:57.340-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:57.383-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:34:57.514-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:57.545-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:34:57.821-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:34:57.821-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:34:57.821-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:34:57.821-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:57.821-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:57.898-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:57.898-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:57.921-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:34:58.043-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:34:58.043-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:34:58.043-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:34:58.043-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:58.043-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:58.119-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:58.121-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:58.133-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:34:58.375-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:58.388-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:34:58.484-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:58.514-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:34:58.720-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:58.737-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:34:58.826-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:58.838-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:34:59.144-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:34:59.144-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:34:59.144-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:34:59.144-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:59.144-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:59.237-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:59.239-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:59.263-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:34:59.306-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:34:59.306-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:34:59.306-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:34:59.306-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:34:59.306-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:34:59.353-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:34:59.353-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:34:59.371-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:34:59.692-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:34:59.722-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:34:59.893-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:34:59.922-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:35:00.262-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:00.301-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:35:00.553-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:00.582-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:35:00.973-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:00.973-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:35:00.973-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:35:00.973-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:00.973-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:01.103-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:01.103-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:01.151-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
PipelineManager 2017-04-25T13:35:01.331-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:56.366966613-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]}
]
CapiNozzle 2017-04-25T13:35:01.355-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:01.355-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:35:01.355-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:35:01.355-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:01.355-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:01.477-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:01.477-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:01.494-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:35:01.590-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:01.605-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:35:01.843-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:35:01.857-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:35:02.135-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:02.161-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:35:02.356-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:35:02.374-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
GenericSupervisor 2017-04-25T13:35:02.571-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:35:02.571-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:35:02.571-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:35:02.739-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:02.739-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:35:02.739-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:35:02.739-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:02.739-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:02.819-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:02.819-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:02.835-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
ToplogyChangeDetector 2017-04-25T13:35:02.845-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:35:03.002-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:35:03.002-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:35:03.002-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:35:03.002-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:03.002-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:03.087-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:03.087-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:03.106-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:35:03.321-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:03.342-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:35:03.667-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:03.678-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:35:03.902-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:03.915-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:35:04.094-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:04.104-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:35:04.827-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:04.827-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:35:04.827-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:35:04.827-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:04.827-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:04.928-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:04.928-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:04.939-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:35:05.170-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:05.170-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:35:05.170-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:35:05.170-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:05.170-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:05.252-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:05.252-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:05.263-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:35:05.387-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:05.398-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:35:05.654-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:35:05.665-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:35:05.843-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:05.855-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:35:06.030-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:35:06.041-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:35:06.261-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:06.261-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:35:06.261-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:35:06.261-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:06.261-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:06.336-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:06.336-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:06.347-05:00 [INFO] Retrying update docs for vb 205 for the 1th time
PipelineManager 2017-04-25T13:35:06.367-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
PipelineManager 2017-04-25T13:35:06.367-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:35:06.367-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:06.367-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
PipelineRuntimeCtx 2017-04-25T13:35:06.367-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:35:06.367-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:06.367-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:35:06.367-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:06.367402835 -0500 CDT
GenericSupervisor 2017-04-25T13:35:06.367-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:06.367-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:35:06.367-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:06.367-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:35:06.367-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:06.367-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:35:06.367-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:35:06.367-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:35:06.367-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:35:06.367-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-537781854 has been stopped
GenericPipeline 2017-04-25T13:35:06.367-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-537781854 is stopped
PipelineManager 2017-04-25T13:35:06.367-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:35:06.368-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:34:56.366966613-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:35:06.368-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:06.368-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
GenericPipeline 2017-04-25T13:35:06.368-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
GenericPipeline 2017-04-25T13:35:06.368-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
GenericPipeline 2017-04-25T13:35:06.368-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-537781854
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:06.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:06.369-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:35:06.369-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:06.369-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:35:06.369-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:06.369-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:35:06.369-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:06.369-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:35:06.369-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:06.369-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:06.369-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:06.369151766 -0500 CDT
DcpNozzle 2017-04-25T13:35:06.369-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:35:06.370-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:06.3695182 -0500 CDT
DcpNozzle 2017-04-25T13:35:06.370-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:35:06.370-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:35:06.370-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:06.370-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:06.370-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:06.370-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:06.371-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:35:06.371-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:06.371-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:35:06.374-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:35:06.374-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:35:06.374-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:06.374257012 -0500 CDT
DcpNozzle 2017-04-25T13:35:06.374-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:06.374276219 -0500 CDT
DcpNozzle 2017-04-25T13:35:06.374-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:35:06.374-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:35:06.374-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:06.374-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:35:06.374-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:06.374-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:35:06 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:35:06 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:35:06 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:35:06 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:35:06.411-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:35:06.412-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:35:06.412-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:06.412-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:35:06.504-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:06.505-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:35:06.505-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:35:06.579-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:06.579-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:35:06.580-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:06.580-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:35:06.580-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:06.580-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:35:06.580-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:35:06.580-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:35:06.580-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
GenericSupervisor 2017-04-25T13:35:06.582-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:35:06.582-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
GenericSupervisor 2017-04-25T13:35:06.583-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:06.583-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:06.583-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:06.583-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:06.583-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:35:06.583-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
CheckpointManager 2017-04-25T13:35:06.583-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
CapiUtils 2017-04-25T13:35:06.614-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:06.614-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:06.614-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:35:06.615-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
StatisticsManager 2017-04-25T13:35:06.615-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:35:06.616-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
XDCRFactory 2017-04-25T13:35:06.616-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:35:06.616-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647 is constructed, start it
GenericPipeline 2017-04-25T13:35:06.617-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[62 157 115 142 27 93 133 145 165 65 143 34 59 71 41 44 53 87 140 141 4 8 153 60 134 111 135 73 103 120 162 1 13 57 15 23 158 7 101 61 116 119 21 48 118 88 95 125 138 170 72 117 160 106 136 29 130 5 84 70 76 28 100 12 19 75 148 33 55 16 114 139 155 168 11 26 167 90 10 52 156 25 104 80 20 43 150 169 36 98 22 35 63 92 99 66 79 6 154 161 56 107 144 164 38 102 86 83 91 128 121 24 64 77 113 45 47 105 108 126 9 49 46 149 127 129 68 97 18 163 96 54 85 122 32 39 146 124 132 42 89 112 0 51 81 82 58 123 30 31 131 3 37 67 151 152 40 50 109 14 94 69 110 159 166 2 78 137 17 74 147] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[307 222 308 314 242 274 316 240 228 189 233 300 334 339 218 333 289 211 213 225 243 220 310 279 321 214 259 327 246 271 298 172 192 202 295 312 209 278 340 198 205 256 286 317 248 303 181 197 221 290 318 296 212 268 280 332 174 210 230 338 276 253 238 254 302 294 304 241 196 287 297 315 266 285 292 267 313 329 177 203 293 180 273 336 234 216 320 322 206 237 239 331 247 272 229 231 250 306 195 194 200 264 275 193 207 277 291 323 175 236 258 324 201 224 176 335 337 227 261 328 217 178 182 265 283 263 232 262 288 260 269 305 185 223 244 171 270 179 188 191 245 186 215 255 325 183 184 219 282 226 249 251 204 301 319 187 208 235 252 299 326 309 173 199 281 284 311 257 330 190]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[21 48 61 116 119 88 95 118 72 117 125 138 170 160 106 136 29 130 5 84 70 76 12 19 28 100 33 55 75 148 16 114 11 26 139 155 168 167 10 52 90 25 104 156 20 43 80 36 98 150 169 22 35 63 66 79 92 99 6 154 56 107 161 38 102 144 164 86 83 91 128 24 64 121 45 47 77 113 9 49 105 108 126 46 127 129 149 68 97 18 163 96 32 39 54 85 122 146 42 89 124 132 0 51 112 81 82 30 31 58 123 3 37 131 67 151 40 50 152 14 94 109 69 110 2 78 159 166 17 74 137 147 62 115 142 157 27 93 133 145 165 34 59 65 143 71 4 8 41 44 53 87 140 141 60 134 153 111 135 1 13 73 103 120 162 15 23 57 7 101 158] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[214 259 327 246 271 298 172 192 202 209 278 295 312 340 198 205 256 286 317 181 197 248 303 221 290 318 296 212 268 280 332 174 210 230 276 338 253 238 254 302 241 294 304 196 287 297 315 266 285 292 267 313 329 177 203 293 180 234 273 336 216 320 322 206 237 239 331 247 272 195 229 231 250 306 193 194 200 264 275 207 277 291 323 175 236 201 224 258 324 176 335 337 217 227 261 328 178 182 265 283 263 232 262 260 288 269 305 171 185 223 244 270 179 188 191 245 186 215 255 325 183 184 219 282 226 249 251 204 301 319 187 208 235 252 299 326 309 173 199 281 284 311 190 257 330 307 222 308 314 242 274 240 316 228 189 233 300 218 334 339 333 289 211 213 225 243 220 310 279 321]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:35:06.617-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647...
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:35:06.626-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:35:06.626-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:06.626-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:35:06.626-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:35:06.626-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:35:06.626-05:00 [INFO] Checkpointing starts in 1800 sec
PipelineRuntimeCtx 2017-04-25T13:35:06.626-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:35:06.626-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:35:06.627-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:35:06.627-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:35:06.629-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:35:06.629-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:35:06.629-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:35:06.629-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:35:06.632-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:35:06.643-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:06.643-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:06.643-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:35:06.657-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:35:06.659-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:35:06.659-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:35:06.659-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
ToplogyChangeDetector 2017-04-25T13:35:06.662-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:35:06.663-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:35:06.663-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:35:06.727-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:35:06.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:35:06.729-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:06.729-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:35:06.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:35:06.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:35:06.730-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:35:06.748-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:35:06.748-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:35:06.748-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:35:06.768-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:35:06.768-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:35:06.833-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:35:07.025-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:35:07.025-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:07.025-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:35:07.055-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:35:07.055-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:35:07.055-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:35:07.055-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647 is started----------
PipelineManager 2017-04-25T13:35:07.055-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:35:07.055-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:35:07.055-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:35:07.101-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:35:07.101-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:35:07.101-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:35:07.101-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:35:07 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 2 successful
CapiNozzle 2017-04-25T13:35:07.203-05:00 [INFO] Send batch count=167
CapiNozzle 2017-04-25T13:35:07.234-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:07.234-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:35:07 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:35:07.242-05:00 [INFO] Send batch count=19
CapiNozzle 2017-04-25T13:35:07.270-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:35:07 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 108 successful
CapiNozzle 2017-04-25T13:35:07.311-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:35:07.354-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:35:07 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 227 successful
CapiNozzle 2017-04-25T13:35:07.572-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:07.572-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:35:07 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 131 successful
CapiNozzle 2017-04-25T13:35:07.592-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:35:07.605-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:35:07.640-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:35:07 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 338 successful
CapiNozzle 2017-04-25T13:35:07.665-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
2017/04/25 13:35:07 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 38 successful
StatisticsManager 2017-04-25T13:35:07.685-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:35:07 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 261 successful
CapiNozzle 2017-04-25T13:35:07.869-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:35:07 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 164 successful
CapiNozzle 2017-04-25T13:35:07.877-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:35:07.877-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:35:07.877-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:35:07.877-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
ReplicationManager 2017-04-25T13:35:07.877-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
CapiNozzle 2017-04-25T13:35:07.887-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
PipelineManager 2017-04-25T13:35:07.906-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:35:07.906-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:35:07.908-05:00 [INFO] err_list=[{"time":"2017-04-25T13:35:07.908562484-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]
PipelineManager 2017-04-25T13:35:07.908-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:35:07 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:35:07.986-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:07.986-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:35:07 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:35:07 UPR_STREAMREQ for vb 69 successful
CapiNozzle 2017-04-25T13:35:07.998-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
2017/04/25 13:35:08 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 188 successful
CapiNozzle 2017-04-25T13:35:08.367-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:08.367-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:35:08.367-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:35:08.367-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:08.367-05:00 [INFO] Send batch count=500
2017/04/25 13:35:08 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 90 successful
CapiNozzle 2017-04-25T13:35:08.492-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:08.492-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:08.507-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:35:08 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 94 successful
StatisticsManager 2017-04-25T13:35:08.637-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:35:08.637-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647 {"Errors": "[{\"time\":\"2017-04-25T13:35:07.908562484-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 17, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 78, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 16424, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 78, "set_filtered": 0, "set_received_from_dcp": 16424, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]", "Status": "Pending"}

2017/04/25 13:35:08 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 299 successful
CheckpointManager 2017-04-25T13:35:08.646-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
CheckpointManager 2017-04-25T13:35:08.646-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:35:08 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:35:08 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:35:08.669-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:08.686-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
DcpNozzle 2017-04-25T13:35:08.732-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:35:08.768-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:35:09.520-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:09.540-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:35:09.698-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:09.716-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:35:09.884-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:09.900-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:35:11.600-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:11.600-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:35:11.600-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:35:11.600-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:11.600-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:11.718-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:11.718-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:11.753-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:35:11.955-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:35:11.955-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:35:11.955-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:35:11.955-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:11.960-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:12.061-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:12.061-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:12.088-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:35:12.114-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:35:12.171-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:35:12.475-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:35:12.498-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:35:12.657-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:35:12.676-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:35:13.003-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:35:13.023-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:35:13.392-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:35:13.392-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:35:13.392-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:35:13.392-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:13.392-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:13.486-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:13.486-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:13.506-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:35:13.618-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:35:13.618-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:35:13.618-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:35:13.618-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:13.618-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:13.681-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:13.681-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:13.695-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:35:13.834-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:35:13.849-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:35:14.360-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:14.371-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:35:14.557-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:35:14.577-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:35:14.764-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:14.780-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:35:15.089-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:35:15.089-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:35:15.089-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:35:15.089-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:15.089-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:15.159-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:15.159-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:15.178-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:35:15.339-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:15.339-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:35:15.339-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:35:15.339-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:15.339-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:15.419-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:15.419-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:15.443-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:35:15.529-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:15.551-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:35:15.991-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:16.006-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:35:16.168-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:16.185-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
PipelineManager 2017-04-25T13:35:16.334-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:35:07.908562484-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]}
]
CapiNozzle 2017-04-25T13:35:16.346-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:16.364-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:35:16.530-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:16.530-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:35:16.530-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:35:16.530-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:16.530-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:16.618-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:16.618-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericSupervisor 2017-04-25T13:35:16.627-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:35:16.627-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:35:16.627-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:35:16.629-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:35:16.830-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:16.830-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:35:16.830-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:35:16.830-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:16.830-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:35:16.877-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:35:16.928-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:16.928-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:16.939-05:00 [INFO] Retrying update docs for vb 307 for the 1th time
CapiNozzle 2017-04-25T13:35:17.076-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:17.086-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:35:17.474-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:35:17.485-05:00 [INFO] Retrying update docs for vb 307 for the 2th time
CapiNozzle 2017-04-25T13:35:17.782-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:17.792-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
PipelineManager 2017-04-25T13:35:17.908-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries 
PipelineManager 2017-04-25T13:35:17.908-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:35:17.908-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:17.909-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
PipelineRuntimeCtx 2017-04-25T13:35:17.909-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:35:17.909-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:35:17.909-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:17.909-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:35:17.909-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:17.909144086 -0500 CDT
GenericSupervisor 2017-04-25T13:35:17.909-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:17.909-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:35:17.909-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:17.909-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:35:17.909-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:17.909-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:35:17.909-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:35:17.909-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:35:17.909-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-580403647 has been stopped
GenericPipeline 2017-04-25T13:35:17.909-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-580403647 is stopped
PipelineManager 2017-04-25T13:35:17.909-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:35:17.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
PipelineManager 2017-04-25T13:35:17.909-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:35:07.908562484-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:35:17.910-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
PipelineManager 2017-04-25T13:35:17.910-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:17.910-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
GenericPipeline 2017-04-25T13:35:17.910-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
GenericPipeline 2017-04-25T13:35:17.910-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-580403647
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:35:17.910-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:35:17.910-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:35:17.910-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:35:17.910-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:17.910-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:35:17.910-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:17.910-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:17.911-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:35:17.911-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:35:17.911-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:35:17.911-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:35:17.911-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:35:17.911-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:17.911009521 -0500 CDT
DcpNozzle 2017-04-25T13:35:17.912-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:35:17.912-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:17.911731139 -0500 CDT
CapiNozzle 2017-04-25T13:35:17.912-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:17.912-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:35:17.912-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.912-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:35:17.912-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:35:17.913-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:35:17.913-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:35:17.913-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:35:17.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:17.913-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:35:17.913-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:35:17.913-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:17.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:35:17.913-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:17.913-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:35:17.913-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:17.913-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:35:17.914-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:35:17.914-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:17.914686733 -0500 CDT
DcpNozzle 2017-04-25T13:35:17.914-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:35:17.914-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:17.914-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:35:17.914-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:35:17.914-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:17.914962881 -0500 CDT
DcpNozzle 2017-04-25T13:35:17.915-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:35:17.915-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:17.915-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:35:17 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:35:17 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:35:17 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:35:17 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:35:18.029-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:18.029-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:35:18.029-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:35:18.073-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:18.073-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:35:18.074-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:18.074-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:35:18.074-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:18.074-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:35:18.074-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:35:18.074-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:35:18.075-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
GenericSupervisor 2017-04-25T13:35:18.077-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:35:18.077-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
GenericSupervisor 2017-04-25T13:35:18.077-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:18.077-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:18.077-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:18.077-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:18.077-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:35:18.077-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
CheckpointManager 2017-04-25T13:35:18.077-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
CapiUtils 2017-04-25T13:35:18.111-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:18.111-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:18.111-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:35:18.112-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
StatisticsManager 2017-04-25T13:35:18.113-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:35:18.113-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
XDCRFactory 2017-04-25T13:35:18.113-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:35:18.114-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481 is constructed, start it
GenericPipeline 2017-04-25T13:35:18.115-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[101 66 150 86 22 40 87 20 93 102 25 130 110 162 58 103 108 149 156 18 105 1 157 62 0 48 73 84 14 167 64 78 142 71 68 33 122 67 97 51 107 124 96 112 163 56 21 28 42 45 139 148 151 170 6 125 155 111 72 74 85 98 52 126 82 129 80 88 131 24 12 59 127 137 158 32 46 26 136 100 4 3 135 169 123 106 41 77 132 153 35 65 76 55 63 81 91 119 161 61 31 120 140 13 164 39 5 79 2 138 49 54 113 34 43 27 152 109 37 114 19 116 165 95 53 118 10 145 36 144 94 23 69 99 134 141 168 15 44 128 8 117 147 160 166 89 83 121 11 90 92 154 30 60 57 104 159 17 75 115 9 38 47 50 29 133 143 146 70 16 7] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[333 198 303 329 336 193 200 217 283 324 337 313 286 295 319 195 299 312 188 322 202 270 300 210 178 204 240 249 305 185 290 220 227 219 244 254 331 179 206 261 266 284 339 247 260 292 323 328 225 184 248 237 294 232 199 298 330 236 226 253 278 314 213 171 223 231 297 315 191 221 233 255 326 186 251 304 316 172 267 252 291 245 307 194 243 218 229 262 334 196 256 279 189 258 272 180 222 235 280 308 216 321 192 207 274 277 281 250 306 173 197 311 246 301 302 318 320 239 211 264 268 325 234 259 285 181 335 212 289 296 203 187 215 293 332 282 182 287 177 205 230 317 209 263 269 275 214 241 309 257 273 288 338 175 201 224 228 238 340 327 183 190 208 265 310 176 242 271 276 174]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[66 101 86 150 87 20 22 40 25 93 102 110 130 162 108 149 156 18 58 103 1 105 62 157 0 84 14 48 73 64 167 71 78 142 68 33 67 122 51 97 96 107 124 56 112 163 21 45 139 148 151 170 6 28 42 111 125 155 85 98 52 72 74 126 80 82 129 24 88 131 12 137 158 32 59 127 26 46 100 136 3 4 135 169 106 123 132 153 35 41 77 55 65 76 91 119 161 61 63 81 140 13 31 120 39 164 2 5 79 49 138 34 54 113 27 43 152 37 109 19 114 95 116 165 10 53 118 36 145 144 94 99 134 141 168 15 23 69 8 44 128 160 166 89 117 147 11 83 121 154 30 90 92 60 159 17 57 104 9 75 115 50 29 38 47 146 70 133 143 7 16] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[333 303 329 336 198 193 200 217 283 313 324 337 286 295 319 299 312 195 188 322 202 270 300 210 204 240 249 305 178 185 290 220 227 244 254 331 219 261 266 284 339 179 206 247 260 323 328 292 225 184 248 237 294 199 232 236 298 330 226 253 278 314 213 231 297 315 171 223 191 221 233 255 326 186 251 304 316 172 267 245 252 291 307 243 194 229 262 334 218 256 279 196 189 258 272 180 222 235 280 308 216 321 192 207 281 250 274 277 306 197 311 173 246 301 302 318 320 239 211 264 268 325 259 285 234 181 335 289 296 212 203 187 215 293 332 282 182 287 230 317 177 205 209 263 269 275 214 241 309 257 273 288 338 224 228 238 340 175 201 327 183 190 208 265 310 176 242 271 276 174]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:35:18.115-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481...
AsyncListener 2017-04-25T13:35:18.123-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:18.123-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:18.123-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:18.124-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
StatisticsManager 2017-04-25T13:35:18.124-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:35:18.128-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:35:18.128-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:35:18.128-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:35:18.128-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
StatisticsManager 2017-04-25T13:35:18.128-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:35:18.128-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:35:18.128-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:35:18.128-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:35:18.128-05:00 [INFO] Checkpointing starts in 1802.55 sec
PipelineRuntimeCtx 2017-04-25T13:35:18.128-05:00 [INFO] Service CheckpointManager has been started
GenericSupervisor 2017-04-25T13:35:18.128-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:35:18.128-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:35:18.129-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:35:18.162-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:18.162-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:18.162-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:35:18.162-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:35:18.162-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:35:18.162-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:35:18.184-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:35:18.185-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:35:18.190-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:35:18.199-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:35:18.199-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:35:18.216-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:35:18.216-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:18.216-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:35:18.216-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:35:18.216-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:35:18.216-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:35:18.252-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:35:18.253-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:35:18.253-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:35:18.276-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:35:18.276-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:35:18.332-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:35:18.444-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:35:18.444-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:18.444-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:35:18.445-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:35:18.445-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:35:18.445-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:35:18.445-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481 is started----------
PipelineManager 2017-04-25T13:35:18.445-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:35:18.445-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:35:18.445-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:35:18.638-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:35:18.639-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:35:18.639-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:35:18.639-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:35:18 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:35:18.769-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:35:18.772-05:00 [INFO] Send batch count=171
2017/04/25 13:35:18 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:35:18.808-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:18.808-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:18.840-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:35:18 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 18 successful
CapiNozzle 2017-04-25T13:35:18.865-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:35:18.874-05:00 [ERROR] Received malformed response from tcp connection vb=207. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f307/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:35:18.874-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err Received malformed response from tcp connection vb=207. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f307/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
2017/04/25 13:35:18 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 209 successful
CapiNozzle 2017-04-25T13:35:18.892-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
2017/04/25 13:35:18 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:35:18 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 243 successful
CapiNozzle 2017-04-25T13:35:19.085-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:35:19.110-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
CapiNozzle 2017-04-25T13:35:19.124-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:35:19.132-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:35:19.138-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
2017/04/25 13:35:19 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 159 successful
CapiNozzle 2017-04-25T13:35:19.149-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:35:19.157-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:35:19.157-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
CapiNozzle 2017-04-25T13:35:19.157-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:35:19.157-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]
ReplicationManager 2017-04-25T13:35:19.157-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]
2017/04/25 13:35:19 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 245 successful
PipelineManager 2017-04-25T13:35:19.191-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
StatisticsManager 2017-04-25T13:35:19.197-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:35:19.199-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:35:19.228-05:00 [INFO] err_list=[{"time":"2017-04-25T13:35:19.228114182-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries"}]
PipelineManager 2017-04-25T13:35:19.228-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:35:19 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:35:19.297-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:19.297-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:19.325-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
2017/04/25 13:35:19 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 277 successful
CapiNozzle 2017-04-25T13:35:19.478-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:35:19.494-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
2017/04/25 13:35:19 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 289 successful
CheckpointManager 2017-04-25T13:35:19.590-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
CheckpointManager 2017-04-25T13:35:19.590-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:35:19 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:35:19 UPR_STREAMREQ for vb 297 successful
DcpNozzle 2017-04-25T13:35:19.718-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:35:19.733-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
DcpNozzle 2017-04-25T13:35:19.749-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:35:19.764-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
CapiNozzle 2017-04-25T13:35:20.087-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:35:20.087-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:35:20.087-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:35:20.087-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:20.087-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:35:20.134-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:35:20.134-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481 {"Errors": "[{\"time\":\"2017-04-25T13:35:19.228114182-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 61, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 13030, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 61, "set_filtered": 0, "set_received_from_dcp": 13030, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:35:20.137-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:20.137-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:20.159-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:35:20.364-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:35:20.390-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
CapiNozzle 2017-04-25T13:35:20.561-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:35:20.594-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:35:20.688-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:35:20.688-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
CapiNozzle 2017-04-25T13:35:20.688-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:35:20.688-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:20.688-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:20.758-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:20.758-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:20.774-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:35:20.822-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:35:20.842-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:35:21.164-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:35:21.190-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:35:21.326-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:35:21.326-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:35:21.326-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:35:21.326-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:21.326-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:21.405-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:21.405-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:21.433-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:35:21.546-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:35:21.559-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:35:21.798-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:35:21.810-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:35:22.037-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:35:22.037-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:35:22.037-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:35:22.037-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:22.038-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:22.096-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:22.096-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:22.120-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:35:22.349-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:35:22.379-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:35:22.533-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:35:22.560-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:35:22.749-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:35:22.749-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:35:22.749-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:35:22.749-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:22.749-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:22.894-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:22.894-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:22.914-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:35:22.968-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:35:22.983-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:35:23.420-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:23.436-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:35:23.622-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:35:23.622-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:35:23.622-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:35:23.622-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:23.622-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:23.690-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:23.690-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:23.704-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:35:23.911-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:23.928-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:35:24.234-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:24.248-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:35:24.437-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:24.437-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:35:24.437-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:35:24.437-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:24.437-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:24.537-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:24.537-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:24.555-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
CapiNozzle 2017-04-25T13:35:24.796-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:24.817-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:35:25.041-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:35:25.060-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
CapiNozzle 2017-04-25T13:35:25.237-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:25.237-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:35:25.237-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:35:25.237-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:25.237-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:25.307-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:25.307-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:25.333-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:35:25.438-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:35:25.451-05:00 [INFO] Retrying update docs for vb 105 for the 3th time
CapiNozzle 2017-04-25T13:35:25.655-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:25.676-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:35:25.805-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:35:25.805-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 105 after 3 retries
CapiNozzle 2017-04-25T13:35:25.805-05:00 [ERROR] Raise error condition batch update docs failed for vb 105 after 3 retries
GenericSupervisor 2017-04-25T13:35:25.805-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:25.805-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:25.890-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:25.890-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:25.907-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:35:26.182-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:26.195-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:35:26.304-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:35:26.315-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:35:26.589-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:35:26.589-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:35:26.589-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:35:26.589-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:26.589-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:26.684-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:26.684-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:26.695-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:35:26.792-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:35:26.803-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:35:27.081-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:35:27.092-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:35:27.396-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:35:27.396-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:35:27.396-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:35:27.396-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:27.396-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:27.479-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:27.479-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:27.490-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:35:27.693-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:35:27.704-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:35:27.978-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:27.989-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
GenericSupervisor 2017-04-25T13:35:28.128-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:35:28.129-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:35:28.129-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:35:28.419-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:35:28.419-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:35:28.419-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:35:28.419-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:28.419-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:35:28.425-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:35:28.523-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:28.524-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:28.534-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:35:28.882-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:35:28.893-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
PipelineManager 2017-04-25T13:35:29.228-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries 
PipelineManager 2017-04-25T13:35:29.228-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:35:29.228-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:29.228-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
PipelineRuntimeCtx 2017-04-25T13:35:29.228-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:35:29.228-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:35:29.228-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:35:29.228-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:35:29.228-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:29.228-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:35:29.229-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:29.22896536 -0500 CDT
GenericSupervisor 2017-04-25T13:35:29.229-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:29.229-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:35:29.229-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:29.229-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:35:29.229-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:29.229-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:35:29.229-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-74407481 has been stopped
GenericPipeline 2017-04-25T13:35:29.229-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-74407481 is stopped
PipelineManager 2017-04-25T13:35:29.229-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:35:29.229-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:35:19.228114182-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 207 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:35:29.229-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:29.229-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
GenericPipeline 2017-04-25T13:35:29.229-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
GenericPipeline 2017-04-25T13:35:29.229-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
GenericPipeline 2017-04-25T13:35:29.230-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-74407481
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:29.230-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:35:29.230-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:29.230-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:35:29.230-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:35:29.230-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:29.230-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:35:29.230-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:29.231-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:35:29.231-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:29.231-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:35:29.231-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:29.231-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:29.231-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:29.230801285 -0500 CDT
DcpNozzle 2017-04-25T13:35:29.231-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:35:29.231-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:29.23112702 -0500 CDT
DcpNozzle 2017-04-25T13:35:29.231-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:35:29.232-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:35:29.232-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:29.232-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:35:29.232-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:29.232-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:35:29.233-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:29.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:29.233-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:35:29.233-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:29.233-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:35:29.239-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:35:29.239-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:35:29 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:35:29 Stream Closed for vb 172 StreamEnd simulated
DcpNozzle 2017-04-25T13:35:29.239-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:29.239178012 -0500 CDT
DcpNozzle 2017-04-25T13:35:29.239-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:35:29.239-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:29.239-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:35:29 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:35:29.239-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:29.239141842 -0500 CDT
DcpNozzle 2017-04-25T13:35:29.261-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:35:29.261-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:29.261-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:35:29.349-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:29.350-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:35:29.350-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:35:29.427-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:29.427-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:35:29.427-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:29.428-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:35:29.428-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:29.428-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:35:29.428-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:35:29.428-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:35:29.429-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
GenericSupervisor 2017-04-25T13:35:29.431-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:35:29.431-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
GenericSupervisor 2017-04-25T13:35:29.431-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:29.431-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:29.431-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:29.431-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:29.431-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:35:29.431-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
CheckpointManager 2017-04-25T13:35:29.431-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
CapiUtils 2017-04-25T13:35:29.472-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:29.472-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:29.472-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:35:29.473-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
StatisticsManager 2017-04-25T13:35:29.473-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:35:29.473-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
XDCRFactory 2017-04-25T13:35:29.473-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:35:29.473-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208 is constructed, start it
GenericPipeline 2017-04-25T13:35:29.475-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[63 127 108 41 132 1 10 25 89 101 152 33 43 134 30 55 71 35 137 169 48 107 144 149 155 5 36 77 64 121 156 46 129 90 126 154 161 167 65 100 28 105 60 109 136 120 70 79 97 117 147 78 87 50 82 91 111 12 21 45 118 53 72 131 102 166 133 0 122 124 13 58 69 39 116 7 31 38 135 18 37 83 4 52 74 106 110 2 32 57 113 56 15 95 44 104 141 73 103 125 148 8 26 130 62 67 146 80 138 170 34 139 119 140 143 159 16 22 51 42 142 123 86 88 23 61 145 6 59 114 84 99 112 150 11 19 47 168 27 160 163 9 66 17 68 165 157 164 3 94 98 162 20 96 151 158 29 49 115 14 54 93 76 85 153 81 92 128 24 40 75] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[248 180 315 332 174 191 199 201 295 325 205 238 285 321 195 221 265 176 235 239 255 300 308 320 212 217 309 193 252 253 298 267 306 197 209 245 188 294 189 183 190 307 324 340 283 331 333 335 185 233 182 213 244 317 288 279 286 234 241 256 272 314 318 260 178 258 330 337 323 249 328 262 237 297 329 251 310 236 257 319 327 242 250 287 290 338 254 275 186 208 229 198 227 266 181 210 224 246 274 264 305 172 211 219 261 280 293 223 303 311 339 273 289 336 296 247 302 263 276 269 173 243 291 304 326 228 270 312 196 284 203 216 240 259 268 334 226 184 187 194 200 278 316 231 271 277 281 175 225 202 215 282 177 179 206 207 220 301 322 218 292 171 204 230 222 299 313 232 192 214]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[266 198 227 224 246 274 181 210 305 264 172 211 280 219 261 293 311 339 223 303 289 336 273 247 296 263 276 302 173 269 304 326 228 243 291 270 312 196 284 240 203 216 334 226 259 268 194 200 278 184 187 316 231 271 277 281 225 175 215 282 202 206 207 220 177 179 301 322 171 204 218 292 222 230 232 299 313 214 192 180 248 315 332 174 199 201 295 191 325 285 321 205 238 265 176 195 221 239 255 300 235 308 320 212 217 309 253 298 193 252 197 267 306 188 209 245 294 183 189 190 307 324 340 333 335 185 233 283 331 182 213 244 317 279 286 288 256 272 234 241 314 260 318 258 330 337 178 323 249 328 262 237 297 329 236 251 310 257 319 327 287 290 338 242 250 186 208 254 275 229] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[44 104 141 73 103 125 8 26 130 148 62 67 146 80 138 170 34 139 119 140 143 16 22 51 159 42 142 123 86 88 23 61 6 59 114 145 112 150 11 19 47 84 99 168 27 160 163 9 66 17 68 165 3 94 98 157 164 20 96 151 162 29 49 115 158 14 54 93 76 85 153 81 92 24 40 75 128 63 127 108 1 10 25 41 132 89 101 33 43 134 152 30 55 71 35 137 48 107 144 169 5 36 77 149 155 64 121 156 46 129 90 126 154 161 167 65 100 28 105 60 109 136 70 79 97 120 117 147 78 87 91 111 12 21 45 50 82 118 53 72 131 102 166 133 0 122 13 58 69 124 7 31 38 39 116 18 37 83 135 4 52 74 106 110 2 32 57 113 56 15 95]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi worker_batch_size:500 doc_batch_size_kb:2048 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:35:29.475-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208...
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:29.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:35:29.485-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:29.485-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:35:29.485-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:35:29.485-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:35:29.485-05:00 [INFO] Checkpointing starts in 1803.659 sec
PipelineRuntimeCtx 2017-04-25T13:35:29.485-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:35:29.485-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:35:29.486-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:35:29.486-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:35:29.487-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:35:29.487-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:35:29.487-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:35:29.488-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:35:29.494-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:35:29.514-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:29.514-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:29.514-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:35:29.522-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:35:29.523-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:35:29.534-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:35:29.534-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:35:29.534-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:35:29.560-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:35:29.560-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:35:29.576-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:35:29.643-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:35:29.647-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:35:29.647-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:29.647-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:35:29.648-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:35:29.648-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:35:29.651-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:35:29.665-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:35:29.665-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:35:29.665-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:35:29.686-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:35:29.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:35:29.857-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:35:29.858-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:29.858-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:35:29.860-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:35:29.860-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:35:29.860-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:35:29.860-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208 is started----------
PipelineManager 2017-04-25T13:35:29.860-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:35:29.860-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:35:29.860-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:35:30.469-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:35:30.469-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:35:30.469-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:35:30.469-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
StatisticsManager 2017-04-25T13:35:30.551-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:35:30 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:35:30.587-05:00 [INFO] Send batch count=47
2017/04/25 13:35:30 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:35:30.620-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:35:30.620-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f204/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:35:30.620-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f204/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:35:30.621-05:00 [ERROR] Received malformed response from tcp connection vb=102. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f100/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:35:30.621-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err Received malformed response from tcp connection vb=102. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f100/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:35:30.640-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:35:30.645-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:35:30 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 108 successful
CapiNozzle 2017-04-25T13:35:30.678-05:00 [INFO] Send batch count=302
2017/04/25 13:35:30 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 12 successful
CapiNozzle 2017-04-25T13:35:30.848-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
2017/04/25 13:35:30 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 218 successful
CapiNozzle 2017-04-25T13:35:30.879-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
2017/04/25 13:35:30 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:35:30 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 339 successful
CapiNozzle 2017-04-25T13:35:31.298-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:31.318-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:35:31.346-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:31.371-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
2017/04/25 13:35:31 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 247 successful
PipelineManager 2017-04-25T13:35:31.461-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:35:31 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:35:31.530-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:35:31.530-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:35:31.530-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:35:31.530-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]
ReplicationManager 2017-04-25T13:35:31.530-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]
StatisticsManager 2017-04-25T13:35:31.538-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
PipelineManager 2017-04-25T13:35:31.539-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:35:31.539-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:35:31.542-05:00 [INFO] err_list=[{"time":"2017-04-25T13:35:31.542132085-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries"}]
PipelineManager 2017-04-25T13:35:31.542-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:35:31 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 262 successful
CapiNozzle 2017-04-25T13:35:31.598-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:31.598-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:31.612-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:35:31 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 268 successful
CapiNozzle 2017-04-25T13:35:31.730-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:31.759-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:35:31 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:35:31 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 194 successful
CapiNozzle 2017-04-25T13:35:32.095-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CheckpointManager 2017-04-25T13:35:32.119-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
CheckpointManager 2017-04-25T13:35:32.119-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:35:32.124-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:35:32 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:35:32 UPR_STREAMREQ for vb 98 successful
DcpNozzle 2017-04-25T13:35:32.273-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:35:32.277-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:35:32.478-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
StatisticsManager 2017-04-25T13:35:32.490-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:35:32.491-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208 {"Errors": "[{\"time\":\"2017-04-25T13:35:31.542132085-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 142, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 4, "docs_received_from_dcp": 39571, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 142, "set_filtered": 0, "set_received_from_dcp": 39571, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:35:32.491-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:35:32.715-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:32.738-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:35:33.529-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:35:33.529-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:35:33.529-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:35:33.529-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:33.529-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:33.619-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:33.619-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:33.662-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:35:33.777-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:35:33.777-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:35:33.777-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:35:33.777-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:35:33.777-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:35:33.879-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:35:33.879-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:35:33.898-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:35:34.246-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:35:34.249-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:35:35.253-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:35:35.292-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
GenericSupervisor 2017-04-25T13:35:43.507-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:35:43.301-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:35:43.358-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries 
GenericSupervisor 2017-04-25T13:35:43.916-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:35:43.908-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:44.094-05:00 [INFO] Finish executing 0x681fb0
PipelineManager 2017-04-25T13:35:44.221-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:44.775-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
PipelineRuntimeCtx 2017-04-25T13:35:44.839-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:35:44.841-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:44.856-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:35:44.870-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:44.842753634 -0500 CDT
GenericSupervisor 2017-04-25T13:35:44.883-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:44.883-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:35:44.885-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:44.887-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:35:44.887-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:35:44.887-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:35:44.889-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:35:44.890-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:35:44.891-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:35:44.891-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-428283208 has been stopped
GenericPipeline 2017-04-25T13:35:44.953-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-428283208 is stopped
PipelineManager 2017-04-25T13:35:44.953-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:35:44.955-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:35:31.542132085-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:35:44.977-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
PipelineManager 2017-04-25T13:35:44.978-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:35:44.980-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
GenericPipeline 2017-04-25T13:35:44.980-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
GenericPipeline 2017-04-25T13:35:44.981-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-428283208
AsyncListener 2017-04-25T13:35:44.982-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:35:44.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:35:44.992-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:44.993-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
GenericPipeline 2017-04-25T13:35:45.031-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:45.031-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:35:45.038-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:35:45.042-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:45.042-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:35:45.042-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:35:45.048-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:35:45.048-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:35:45.048-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:35:45.048-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:35:45.049-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:35:45.010-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:35:45.060-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:35:45.068-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:45.040478658 -0500 CDT
DcpNozzle 2017-04-25T13:35:45.070-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:35:45.070-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:45.04875346 -0500 CDT
CapiNozzle 2017-04-25T13:35:45.104-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:35:45.104-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:45.105-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:35:45.105-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:35:45.109-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:35:45.109-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:35:45.109-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:35:45.112-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:35:45.112-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:45.116-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:35:45.116-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:35:45.116-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:35:45.924-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:35:46.813-05:00 [INFO] Executing 0x65e930 timed out
DcpNozzle 2017-04-25T13:35:48.998-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:48.985769639 -0500 CDT
GenericPipeline 2017-04-25T13:35:46.869-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:35:50.680-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:35:51.012-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
DcpNozzle 2017-04-25T13:35:51.020-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:35:51.028-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:35:51.007-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:35:51.092-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:35:51.092-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericSupervisor 2017-04-25T13:35:49.438-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:35:51.115-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
DcpNozzle 2017-04-25T13:35:51.119-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:35:51.032-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:35:51.023137434 -0500 CDT
DcpNozzle 2017-04-25T13:35:51.155-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericSupervisor 2017-04-25T13:35:51.150-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:35:54.750-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:35:31.542132085-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:35:56.945-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:56.983-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:35:56.997-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
ReplicationManager 2017-04-25T13:35:57.044-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
XDCRFactory 2017-04-25T13:35:58.019-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:35:58.050-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:35:58.076-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:58.076-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:35:58.076-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:35:58.076-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:35:58.077-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:35:58.086-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:35:58.101-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
GenericSupervisor 2017-04-25T13:35:58.113-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:35:58.114-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
GenericSupervisor 2017-04-25T13:35:58.115-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:58.115-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:58.115-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:58.115-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:35:58.115-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:35:58.119-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
CheckpointManager 2017-04-25T13:35:58.119-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
CapiUtils 2017-04-25T13:35:58.236-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:58.238-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:58.240-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:35:58.245-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
StatisticsManager 2017-04-25T13:35:58.264-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:35:58.266-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
XDCRFactory 2017-04-25T13:35:58.267-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:35:58.270-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 is constructed, start it
GenericPipeline 2017-04-25T13:35:58.277-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[134 3 5 33 44 130 2 47 99 20 46 93 74 100 162 15 25 26 38 70 4 6 92 128 48 140 49 76 78 31 108 22 80 98 164 23 55 12 18 60 63 67 166 167 17 136 157 11 62 132 146 10 113 114 126 45 117 53 122 102 110 148 52 101 135 161 152 82 90 16 35 95 118 57 66 139 109 165 43 84 125 168 9 115 138 79 73 97 150 159 13 21 41 106 83 170 39 50 37 127 103 149 133 156 40 111 68 96 72 89 105 121 124 144 19 30 75 88 160 34 7 71 8 112 131 14 24 69 141 153 1 27 58 29 151 143 104 51 42 119 28 54 163 87 120 154 123 137 147 81 91 86 129 158 36 145 32 65 85 142 169 107 0 64 94 155 56 59 61 77 116] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[190 224 271 314 311 273 308 336 193 195 173 180 256 329 248 172 215 262 338 183 240 254 186 184 185 235 245 194 264 302 228 261 234 257 202 306 226 244 318 337 237 210 217 221 255 269 287 206 296 241 227 276 270 317 289 174 279 178 188 232 301 229 253 334 220 230 307 199 239 304 197 201 175 310 238 290 293 298 218 251 259 326 246 295 291 323 260 275 277 211 216 243 250 266 179 200 196 288 299 300 324 325 205 209 320 335 191 213 294 305 315 330 189 283 331 177 181 207 263 280 328 278 282 285 312 322 340 198 281 292 208 268 265 316 182 203 309 339 333 222 242 327 233 313 284 236 247 332 267 303 321 219 258 272 274 297 176 187 223 249 286 319 171 225 214 231 252 204 212 192]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[238 290 293 298 310 218 251 259 326 246 295 260 275 277 291 323 211 216 243 250 266 179 200 196 288 299 300 324 325 205 209 320 191 213 294 335 189 283 305 315 330 331 177 181 207 280 328 263 278 282 285 312 198 281 292 322 340 208 268 265 316 182 203 309 339 333 222 242 233 313 327 236 247 284 267 303 321 332 219 258 272 274 297 176 187 223 286 319 249 171 225 204 212 214 231 252 192 190 224 314 271 273 308 311 193 195 336 180 256 329 173 172 215 248 262 338 183 240 254 186 184 185 235 245 194 264 302 228 261 234 257 202 306 226 244 318 337 210 217 237 221 206 255 269 287 296 241 227 276 270 289 317 174 178 188 232 279 301 229 220 253 334 230 307 199 239 304 175 197 201] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[39 50 127 37 149 103 156 133 111 40 96 68 89 105 121 124 72 144 30 75 88 160 19 34 71 7 112 131 8 24 69 141 153 14 1 58 27 151 29 143 104 51 119 42 54 163 28 120 154 87 137 147 123 91 81 129 158 86 145 36 65 85 142 169 32 107 64 94 0 59 61 77 116 155 56 5 33 44 130 134 3 47 99 2 46 93 20 25 26 38 70 74 100 162 15 6 92 4 128 140 48 76 49 78 108 31 22 98 164 80 55 23 18 12 63 67 166 167 60 136 157 17 62 132 146 11 113 114 126 10 117 45 122 53 110 148 102 101 135 161 52 152 90 82 35 95 118 16 66 139 57 165 109 84 125 168 43 115 138 9 79 97 150 159 73 21 41 106 13 170 83]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 replication_type:capi filter_expression: active:true optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info stats_interval:1000 checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10]
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:35:58.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:35:58.292-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:35:58.294-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:35:58.294-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:35:58.303-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913...
GenericSupervisor 2017-04-25T13:35:58.299-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:35:58.295-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:35:58.325-05:00 [INFO] Checkpointing starts in 1803.921 sec
PipelineRuntimeCtx 2017-04-25T13:35:58.328-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:35:58.330-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:35:58.337-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:35:58.389-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:35:58.389-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:35:58.390-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:35:58.423-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:35:58.424-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:35:58.424-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:35:58.425-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:35:58.441-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:35:58.480-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:35:58.481-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:35:58.483-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:35:58.506-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:35:58.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:35:58.531-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:35:58.546-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:35:58.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:35:58.699-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:35:58.702-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:58.703-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:35:58.755-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:35:58.757-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:35:58.762-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:35:58.789-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:35:58.789-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:35:58.789-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:35:58.797-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:35:58.802-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:35:58.802-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:35:58.985-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:35:58.986-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:35:58.986-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:35:59.006-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:35:59.006-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:35:59.007-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:35:59.007-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 is started----------
PipelineManager 2017-04-25T13:35:59.009-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:35:59.009-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:35:59.010-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:35:59.140-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:35:59.150-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:35:59.150-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:35:59.155-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:35:59 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 306 successful
StatisticsManager 2017-04-25T13:35:59.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:35:59 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 326 successful
CapiNozzle 2017-04-25T13:35:59.987-05:00 [INFO] Send batch count=100
2017/04/25 13:35:59 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:35:59 UPR_STREAMREQ for vb 130 successful
CapiNozzle 2017-04-25T13:36:00.016-05:00 [INFO] Send batch count=44
2017/04/25 13:36:00 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 60 successful
StatisticsManager 2017-04-25T13:36:00.481-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:36:00 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 176 successful
CheckpointManager 2017-04-25T13:36:00.644-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
CheckpointManager 2017-04-25T13:36:00.644-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:36:00 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:36:00 UPR_STREAMREQ for vb 291 successful
DcpNozzle 2017-04-25T13:36:00.768-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:36:00.810-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
ToplogyChangeDetector 2017-04-25T13:36:09.004-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:36:08.986-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:36:09.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
GenericSupervisor 2017-04-25T13:36:09.874-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:36:10.003-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:36:10.072-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:36:10.075-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:36:10.047-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:36:10.147-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:36:10.151-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:36:10.207-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:36:10.225-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err read tcp 10.202.98.33:9091: i/o timeout.
StatisticsManager 2017-04-25T13:36:10.296-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:35:31.542132085-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 102 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 1250, "dcp_dispatch_time": 643, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 64, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 29173, "docs_rep_queue": 30278, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 20807, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 64, "set_filtered": 0, "set_received_from_dcp": 29958, "size_rep_queue": 40116904, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 23}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 42, "docs_opt_repd": 0, "docs_rep_queue": 21290, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 42, "size_rep_queue": 28379396, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 78, "max": 8, "mean": 8, "min": 8}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 22, "docs_opt_repd": 0, "docs_rep_queue": 15431, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 22, "size_rep_queue": 20043944, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 33, "max": 60, "mean": 60, "min": 60}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 171, "dcp_dispatch_time": {"count": 16675, "max": 790140, "mean": 798.996, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 24586, "expiry_received_from_dcp": 0, "set_received_from_dcp": 24586}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 11438, "max": 214598, "mean": 414.865, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 19005, "expiry_received_from_dcp": 0, "set_received_from_dcp": 19009}}

StatisticsManager 2017-04-25T13:36:10.297-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 24586 items, sent 100 items
StatisticsManager 2017-04-25T13:36:10.297-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 19221 items, sent 44 items
StatisticsManager 2017-04-25T13:36:10.305-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:24980]
StatisticsManager 2017-04-25T13:36:10.312-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 24980 items, sent 24980 items.
StatisticsManager 2017-04-25T13:36:10.312-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:19221]
StatisticsManager 2017-04-25T13:36:10.312-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 19221 items, sent 19221 items.
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:10.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
CapiNozzle 2017-04-25T13:36:10.323-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:36:10.351-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:36:10.637-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
StatisticsManager 2017-04-25T13:36:10.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:36:10.689-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
PipelineManager 2017-04-25T13:36:10.741-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
StatisticsManager 2017-04-25T13:36:10.768-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:36:10.785-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:36:10.853-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:36:10.861-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:36:10.938-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:36:12.062-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:36:12.041-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:36:12.874-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:36:13.048-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:36:13.050-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:36:13.252-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
CapiNozzle 2017-04-25T13:36:13.291-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
StatisticsManager 2017-04-25T13:36:13.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
StatisticsManager 2017-04-25T13:36:13.577-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:36:13.621-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 976, "dcp_dispatch_time": 203, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 64, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 65623, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 64, "set_filtered": 0, "set_received_from_dcp": 65623, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

PipelineManager 2017-04-25T13:36:13.623-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:36:13.626-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:36:13.668-05:00 [INFO] err_list=[{"time":"2017-04-25T13:36:13.668252554-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:36:13.711-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:36:13.811-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:36:13.811-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:36:13.811-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:36:13.811-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:36:13.811-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:36:14.267-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:36:14.281-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:36:14.697-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:36:14.713-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:36:14.770-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:36:14.857-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
GenericSupervisor 2017-04-25T13:36:19.764-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:36:19.767-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
ToplogyChangeDetector 2017-04-25T13:36:19.744-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:36:19.825-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:36:19.966-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:36:20.152-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:36:20.756-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:36:21.240-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:36:23.731-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
PipelineManager 2017-04-25T13:36:23.802-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:36:23.884-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:36:23.888-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:36:23.891-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
PipelineRuntimeCtx 2017-04-25T13:36:23.894-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:36:23.897-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:36:23.906-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:36:23.937-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:36:23.898319124 -0500 CDT
GenericSupervisor 2017-04-25T13:36:24.073-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:36:24.073-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:36:24.077-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:36:24.078-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:36:24.081-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:36:24.081-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:36:24.084-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:36:24.084-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:36:24.091-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:36:24.091-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 has been stopped
CapiNozzle 2017-04-25T13:36:24.148-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
GenericPipeline 2017-04-25T13:36:24.193-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-77822913 is stopped
PipelineManager 2017-04-25T13:36:24.193-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:36:24.194-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:36:13.668252554-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:36:24.224-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:36:24.226-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
GenericPipeline 2017-04-25T13:36:24.231-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
GenericPipeline 2017-04-25T13:36:24.231-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
GenericPipeline 2017-04-25T13:36:24.231-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-77822913
AsyncListener 2017-04-25T13:36:24.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:36:24.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:36:24.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:36:24.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:36:24.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:36:24.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:36:24.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:36:24.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:36:24.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:36:24.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:36:24.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:36:24.241-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:36:24.242-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:36:24.242-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:36:24.247-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:36:24.250-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:36:24.251-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:36:24.254-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:36:24.271-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:36:24.271-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:36:24.271-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:36:24.278-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:36:24.249941983 -0500 CDT
DcpNozzle 2017-04-25T13:36:24.297-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:36:24.297-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:36:24.271715848 -0500 CDT
GenericPipeline 2017-04-25T13:36:24.297-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:36:24.297-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:36:24.297-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:36:24.298-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:36:24.305-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:36:24.305-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:36:24.306-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:36:24.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:36:24.308-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:36:24.308-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:36:24.309-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:36:24.313-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:36:24.314-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:36:24.314-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:36:24.314-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:36:24.334-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:36:24.336-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:36:24.336-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:36:24.336-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:36:24.539-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:36:24.654-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:36:26.054-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:36:26.044288528 -0500 CDT
DcpNozzle 2017-04-25T13:36:26.041-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:36:26.011742353 -0500 CDT
DcpNozzle 2017-04-25T13:36:26.379-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:36:26.171-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:36:26.387-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:36:26.396-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
DcpNozzle 2017-04-25T13:36:26.361-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:36:26.311-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:36:26.791-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:36:26.576-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:36:26.795-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:36:26.846-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
PipelineManager 2017-04-25T13:36:27.857-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:36:13.668252554-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:36:27.923-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:36:27.937-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:36:27.941-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:36:28.110-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:36:28.241-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:36:28.491-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:36:28.585-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:36:28.648-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:36:28.660-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:36:28.929-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:36:29.040-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:36:29.083-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
GenericSupervisor 2017-04-25T13:36:29.194-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:36:29.206-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
GenericSupervisor 2017-04-25T13:36:29.208-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:36:29.208-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:36:29.210-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:36:29.210-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:36:29.210-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:36:29.218-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
CheckpointManager 2017-04-25T13:36:29.218-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
CapiUtils 2017-04-25T13:36:29.758-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:36:29.785-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:36:29.811-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:36:29.892-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
StatisticsManager 2017-04-25T13:36:29.950-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:36:29.952-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
XDCRFactory 2017-04-25T13:36:29.988-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:36:29.994-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 is constructed, start it
GenericPipeline 2017-04-25T13:36:30.006-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[206 305 188 179 218 228 239 328 332 190 214 263 303 185 276 285 273 182 264 227 230 300 294 312 192 200 256 266 258 175 216 265 336 171 207 219 245 261 331 176 255 335 229 302 243 284 174 290 246 318 205 212 234 330 248 334 225 177 183 189 252 268 296 327 240 333 209 172 223 281 288 184 267 316 309 221 224 278 282 319 237 238 283 199 202 244 315 308 262 270 233 241 193 196 220 222 191 340 242 310 314 231 298 307 251 286 293 299 215 250 323 329 198 204 338 277 321 325 269 274 295 289 320 180 210 203 194 297 253 173 271 324 317 339 226 235 272 275 326 257 287 306 322 178 247 254 201 213 259 187 311 208 197 279 249 313 337 211 232 181 195 217 280 301 304 260 292 186 291 236] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[83 132 137 94 136 39 48 54 71 124 126 139 4 130 168 92 120 114 167 44 62 66 79 131 157 38 42 46 52 29 143 101 133 57 86 156 24 89 122 151 40 64 5 76 140 23 91 147 154 15 99 41 98 117 13 142 169 150 3 10 47 138 8 22 84 128 80 45 73 74 77 1 102 164 27 59 104 43 115 9 103 12 34 63 107 153 162 2 65 155 28 36 119 108 148 49 67 70 87 60 152 37 113 146 159 7 25 125 53 127 160 161 109 163 0 17 32 95 135 145 170 21 35 61 118 33 105 11 129 31 85 158 14 50 93 123 30 68 144 134 51 82 96 111 116 166 18 56 100 106 58 72 6 19 26 69 88 97 149 16 55 112 121 110 141 90 165 20 75 78 81]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[158 14 50 93 123 30 68 144 134 51 82 96 111 116 166 18 56 100 106 58 72 6 19 26 69 88 97 149 16 55 112 121 110 141 90 165 20 75 78 81 83 132 137 94 136 39 48 54 71 124 126 139 4 130 168 92 120 114 167 44 62 66 79 131 157 38 42 46 52 29 143 101 133 57 86 156 24 89 122 151 40 64 5 76 140 23 91 147 154 15 99 41 98 117 13 142 169 150 3 10 47 138 8 22 84 128 80 45 73 74 77 1 102 164 27 59 104 43 115 9 103 12 34 63 107 153 162 2 65 155 28 36 119 108 148 49 67 70 87 60 152 37 113 146 159 7 25 125 53 127 160 161 109 163 0 17 32 95 135 145 170 21 35 61 118 33 105 11 129 31 85] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[173 271 324 317 339 226 235 272 275 326 257 287 306 322 178 247 254 201 213 259 187 311 208 197 279 249 313 337 211 232 181 195 217 280 301 304 260 292 186 291 236 206 305 188 179 218 228 239 328 332 190 214 263 303 185 276 285 273 182 264 227 230 300 294 312 192 200 256 266 258 175 216 265 336 171 207 219 245 261 331 176 255 335 229 302 243 284 174 290 246 318 205 212 234 330 248 334 225 177 183 189 252 268 296 327 240 333 209 172 223 281 288 184 267 316 309 221 224 278 282 319 237 238 283 199 202 244 315 308 262 270 233 241 193 196 220 222 191 340 242 310 314 231 298 307 251 286 293 299 215 250 323 329 198 204 338 277 321 325 269 274 295 289 320 180 210 203 194 297 253]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi filter_expression: active:true worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:36:30.027-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610...
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:36:30.158-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:36:30.164-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:36:30.167-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:36:30.167-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:36:30.167-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:36:30.167-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:36:30.176-05:00 [INFO] Checkpointing starts in 1800.38 sec
PipelineRuntimeCtx 2017-04-25T13:36:30.178-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:36:30.179-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:36:30.180-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:36:30.220-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:36:30.221-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:36:30.221-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:36:30.220-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:36:30.237-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:36:30.264-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:36:30.264-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:36:30.265-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:36:30.292-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:36:30.293-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:36:30.294-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
ToplogyChangeDetector 2017-04-25T13:36:30.299-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:36:30.301-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:36:30.303-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiNozzle 2017-04-25T13:36:30.303-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:36:30.344-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:36:30.469-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:36:30.505-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:36:30.508-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:36:30.508-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:36:30.517-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:36:30.519-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:36:30.524-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:36:30.573-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:36:30.573-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:36:30.573-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:36:30.589-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:36:30.589-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:36:30.818-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:36:30.818-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:36:30.818-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:36:30.818-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:36:30.818-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:36:30.818-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:36:30.819-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 is started----------
PipelineManager 2017-04-25T13:36:30.821-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:36:30.821-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:36:30.822-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:36:30.950-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:36:30.957-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:36:30.957-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:36:30.957-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:36:31 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 204 successful
StatisticsManager 2017-04-25T13:36:31.349-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:36:31 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:36:31 UPR_STREAMREQ for vb 111 successful
CapiNozzle 2017-04-25T13:36:31.435-05:00 [INFO] Send batch count=227
CapiNozzle 2017-04-25T13:36:31.484-05:00 [INFO] Send batch count=464
2017/04/25 13:36:34 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:36:35 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:36:36 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:36:36 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:36:37 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:36:38 UPR_STREAMREQ for vb 20 successful
GenericSupervisor 2017-04-25T13:36:38.884-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:36:39.674-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:36:39.705-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:36:39.709-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:36:40 UPR_STREAMREQ for vb 21 successful
ToplogyChangeDetector 2017-04-25T13:36:41.000-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:36:40.899-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:36:41.436-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:36:41.617-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:36:40 UPR_STREAMREQ for vb 218 successful
GenericSupervisor 2017-04-25T13:36:41.753-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:36:45.830-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:36:46.317-05:00 [INFO] ****************************
CapiNozzle 2017-04-25T13:36:43.607-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
StatisticsManager 2017-04-25T13:36:46.269-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:36:46.827-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:36:49.446-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:36:49.618-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:36:47.050-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:36:49 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:36:49 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:36:49 UPR_STREAMREQ for vb 22 successful
StatisticsManager 2017-04-25T13:36:49.961-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:36:13.668252554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1000, "dcp_dispatch_time": 506, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 384, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 10229, "docs_rep_queue": 6932, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 10229, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 384, "set_filtered": 0, "set_received_from_dcp": 10229, "size_rep_queue": 9346960, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 3893}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 284, "docs_opt_repd": 0, "docs_rep_queue": 6648, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 284, "size_rep_queue": 8844328, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 385, "max": 3916, "mean": 3915.475324675325, "min": 3915}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 100, "docs_opt_repd": 0, "docs_rep_queue": 9060, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 100, "size_rep_queue": 10712903, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 191, "max": 3897, "mean": 3849.696335078534, "min": 3836}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 953, "dcp_dispatch_time": {"count": 3846, "max": 590424, "mean": 615.891, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 9435, "expiry_received_from_dcp": 0, "set_received_from_dcp": 9435}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 20, "dcp_dispatch_time": {"count": 6383, "max": 424193, "mean": 440.474, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 10577, "expiry_received_from_dcp": 0, "set_received_from_dcp": 10577}}

StatisticsManager 2017-04-25T13:36:50.045-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 10882 items, sent 464 items
StatisticsManager 2017-04-25T13:36:50.045-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 10754 items, sent 227 items
StatisticsManager 2017-04-25T13:36:50.050-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:10882]
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 10882 items, sent 10882 items. streams inactive: [95 144 146 47 34 61 87 31 81 96 148 24 72 74 98 163 165 57 63 77 129 139 155 168 54 55 53 73 125 152 161 36 134 79 65 52 76 44 43 121 60 23 128 147 131 150 91 132 133 149 33 64 49 151 75 88 90 46 56 137 71 160 141 142 143 145 69 39 67 127 156 30 25 40 62 68 82 89 122 50 80 135 41 78 99 153 166 28 32 38 85 154 26 130 157 167 93 84 123 138 159 42 83 158 94 97 124 136 169 27 37 170 66 35 45 51 58 59 92 29 162 164 70 86 126 140 48]
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:11508]
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 11510 items, sent 11509 items. streams inactive: [224 250 265 320 325 198 220 334 340 285 291 332 240 251 219 229 254 258 266 269 278 318 172 241 246 253 267 283 294 176 238 319 324 188 196 226 257 288 177 180 184 256 323 239 271 268 275 171 231 237 336 182 230 181 228 225 243 280 292 192 262 263 277 290 293 173 178 331 261 272 333 337 338 339 227 248 327 330 175 235 297 326 249 289 276 298 260 222 299 179 191 234 242 286 328 197 221 232 193 335 186 296 329 236 279 233 321 194 247 187 264 174 185 244 273 190 245 284 287 195 199 252 274 223 255 259 281 295 322 189 183 317 270 282]
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =948 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =949 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:50.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
2017/04/25 13:36:50 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 121 successful
CapiNozzle 2017-04-25T13:36:50.164-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:36:50 UPR_STREAMREQ for vb 318 successful
GenericSupervisor 2017-04-25T13:36:50.257-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:36:50.199-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:36:50.288-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:36:50 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:36:50 UPR_STREAMREQ for vb 125 successful
CapiNozzle 2017-04-25T13:36:50.392-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
StatisticsManager 2017-04-25T13:36:50.436-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
StatisticsManager 2017-04-25T13:36:53.671-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:36:53.460-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:36:53.710-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:36:53.713-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:36:53 UPR_STREAMREQ for vb 321 successful
PipelineManager 2017-04-25T13:36:53.831-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:36:54 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 127 successful
CapiNozzle 2017-04-25T13:36:54.491-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:36:54.521-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:36:13.668252554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 973, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 384, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 29837, "docs_rep_queue": 22578, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 6234, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 384, "set_filtered": 0, "set_received_from_dcp": 29837, "size_rep_queue": 26410118, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 3893}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 284, "docs_opt_repd": 0, "docs_rep_queue": 15456, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 284, "size_rep_queue": 18185383, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 385, "max": 3916, "mean": 3915.475324675325, "min": 3915}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 100, "docs_opt_repd": 0, "docs_rep_queue": 16835, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 100, "size_rep_queue": 19528328, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 191, "max": 3897, "mean": 3849.696335078534, "min": 3836}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 460, "dcp_dispatch_time": {"count": 14600, "max": 90, "mean": 8.106, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 16163, "expiry_received_from_dcp": 0, "set_received_from_dcp": 16163}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 658, "dcp_dispatch_time": {"count": 15236, "max": 83, "mean": 9.273, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 17085, "expiry_received_from_dcp": 0, "set_received_from_dcp": 17085}}

StatisticsManager 2017-04-25T13:36:54.707-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 17564 items, sent 464 items
StatisticsManager 2017-04-25T13:36:54.707-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 17305 items, sent 227 items
StatisticsManager 2017-04-25T13:36:54.833-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:17851]
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 17851 items, sent 17851 items. streams inactive: [96 148 72 81 98 163 165 57 74 77 129 139 155 168 54 55 63 73 152 161 36 53 79 134 52 65 44 76 43 60 128 147 150 131 132 91 133 149 33 64 49 151 88 90 46 75 56 137 71 160 142 143 145 69 141 67 127 156 30 39 40 62 68 82 89 80 135 41 50 99 153 166 28 32 78 85 154 38 157 167 93 130 138 159 42 83 84 158 97 136 169 37 94 170 66 45 51 58 59 92 29 35 162 164 86 140 48 70 144 146 47 95 61 87 31 34]
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:17580]
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 17580 items, sent 17580 items. streams inactive: [236 279 296 329 233 194 247 174 185 187 264 190 244 273 195 199 245 284 287 252 274 189 223 255 259 281 295 183 270 282 198 224 250 265 325 334 340 240 285 291 332 229 251 172 254 258 266 269 278 176 238 241 246 253 267 283 294 324 177 180 188 196 226 257 288 184 256 239 271 323 171 231 268 275 182 230 237 336 181 228 192 225 243 280 292 173 178 262 263 277 290 293 331 227 261 272 333 337 338 339 175 235 248 327 330 249 289 297 326 260 276 298 179 191 299 197 234 242 286 328 193 232 186 335]
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =275 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =276 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:36:54.836-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
2017/04/25 13:36:54 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:36:54 UPR_STREAMREQ for vb 324 successful
CapiNozzle 2017-04-25T13:36:56.428-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:36:56 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:36:57 UPR_STREAMREQ for vb 128 successful
CapiNozzle 2017-04-25T13:36:56.657-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:36:58 UPR_STREAMREQ for vb 28 successful
StatisticsManager 2017-04-25T13:36:58.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:37:00.736-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:37:00.843-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
GenericSupervisor 2017-04-25T13:37:00.884-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:37:01.080-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:37:01.167-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:37:01 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:37:01 UPR_STREAMREQ for vb 129 successful
ToplogyChangeDetector 2017-04-25T13:37:02.667-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:37:02.620-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:37:02.820-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:37:02.828-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:37:03 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:37:04 UPR_STREAMREQ for vb 325 successful
StatisticsManager 2017-04-25T13:37:04.572-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:37:04 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:37:05 UPR_STREAMREQ for vb 130 successful
CapiNozzle 2017-04-25T13:37:06.014-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:37:08.743-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
CapiNozzle 2017-04-25T13:37:10.837-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:37:10.620-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
GenericSupervisor 2017-04-25T13:37:09.945-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:37:11.082-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:37:11.083-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:37:11.097-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:37:46.344-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:37:46.358-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:37:46.361-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:37:46.258-05:00 [ERROR] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 - Connection reset failed
CapiNozzle 2017-04-25T13:37:46.393-05:00 [ERROR] Raise error condition dial tcp 10.202.98.33:9091: i/o timeout
GenericSupervisor 2017-04-25T13:37:46.396-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout]
CAPIService 2017-04-25T13:37:46.479-05:00 [INFO] Received error when making rest call. baseURL=10.202.98.33:9091, path=_pre_replicate, ret_err=Post http://10.202.98.33:9091/_pre_replicate: http: can't write HTTP request on broken connection, statusCode=0, num_retry=5
CAPIService 2017-04-25T13:37:46.325-05:00 [INFO] Received error when making rest call. baseURL=10.202.98.33:9091, path=_pre_replicate, ret_err=Post http://10.202.98.33:9091/_pre_replicate: EOF, statusCode=0, num_retry=5
GenericSupervisor 2017-04-25T13:37:46.332-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
ReplicationManager 2017-04-25T13:37:46.411-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout]
GenericSupervisor 2017-04-25T13:37:46.528-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:37:46.531-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:37:46.547-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:37:46.547-05:00 [INFO] Finish executing 0x681fb0
StatisticsManager 2017-04-25T13:37:46.637-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:36:13.668252554-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1228, "dcp_dispatch_time": 666, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 384, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 41862, "docs_rep_queue": 39361, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 6431, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 384, "set_filtered": 0, "set_received_from_dcp": 41862, "size_rep_queue": 48320327, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 3893}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 284, "docs_opt_repd": 0, "docs_rep_queue": 24507, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 284, "size_rep_queue": 32150504, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 385, "max": 3916, "mean": 3915.475324675325, "min": 3915}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 100, "docs_opt_repd": 0, "docs_rep_queue": 25922, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 100, "size_rep_queue": 32126727, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 191, "max": 3897, "mean": 3849.696335078534, "min": 3836}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 999, "dcp_dispatch_time": {"count": 20864, "max": 59264, "mean": 82.215, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 25090, "expiry_received_from_dcp": 0, "set_received_from_dcp": 25090}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 290, "dcp_dispatch_time": {"count": 20996, "max": 1211730, "mean": 1247.756, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 26287, "expiry_received_from_dcp": 0, "set_received_from_dcp": 26287}}

StatisticsManager 2017-04-25T13:37:46.637-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 25107 items, sent 464 items
StatisticsManager 2017-04-25T13:37:46.637-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 26367 items, sent 227 items
StatisticsManager 2017-04-25T13:37:46.637-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:25107]
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 25108 items, sent 25107 items. streams inactive: [155 168 54 55 63 77 139 161 36 53 73 152 79 134 52 65 44 76 43 60 147 131 150 91 132 33 64 133 149 49 151 46 75 88 90 56 137 71 160 69 141 142 143 145 156 39 67 68 82 40 62 89 41 50 80 135 32 78 99 153 166 38 85 154 93 157 167 159 42 83 84 138 158 136 169 37 94 97 170 66 59 92 35 45 51 58 162 164 140 48 70 86 47 95 144 146 31 34 61 87 72 81 96 148 163 165 57 74 98]
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:26367]
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 26368 items, sent 26367 items. streams inactive: [291 332 240 285 229 251 258 266 269 278 172 254 246 253 267 283 294 176 238 241 196 226 257 288 177 180 188 184 256 239 271 275 171 231 268 336 182 230 237 181 228 243 280 292 192 262 263 277 290 293 173 178 331 272 333 337 338 339 227 261 327 330 175 235 248 326 249 289 297 298 260 276 299 179 191 242 286 328 197 234 193 232 335 186 329 236 279 296 233 194 247 264 174 185 187 244 273 190 245 284 287 195 199 274 252 255 259 281 295 189 183 270 282 250 265 198 334 340]
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =29 
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:37:46.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:37:46.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:37:46.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:37:46.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =18 
StatisticsManager 2017-04-25T13:37:46.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:37:46.642-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:37:46.643-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =81 
StatisticsManager 2017-04-25T13:37:46.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =81 
StatisticsManager 2017-04-25T13:37:46.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:37:46.644-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:37:46.671-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:37:46.689-05:00 [INFO] err_list=[{"time":"2017-04-25T13:37:46.689573685-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout"}]
PipelineManager 2017-04-25T13:37:46.700-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:37:46 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:37:46 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:37:46 UPR_STREAMREQ for vb 227 successful
StatisticsManager 2017-04-25T13:37:46.742-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-909905610 {"Errors": "[{\"time\":\"2017-04-25T13:37:46.689573685-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1228, "dcp_dispatch_time": 666, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 384, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 41862, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 384, "set_filtered": 0, "set_received_from_dcp": 41862, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

CapiNozzle 2017-04-25T13:37:46.795-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:37:47 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:37:51 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:37:53 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:37:53 UPR_STREAMREQ for vb 228 successful
ReplicationManager 2017-04-25T13:37:54.327-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
ToplogyChangeDetector 2017-04-25T13:37:54.414-05:00 [INFO] Pipeline is no longer running, exit.
DcpNozzle 2017-04-25T13:37:53.755-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
DcpNozzle 2017-04-25T13:37:54.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
2017/04/25 13:37:55 UPR_STREAMREQ for vb 32 successful
GenericSupervisor 2017-04-25T13:37:55.258-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:37:55.266-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
2017/04/25 13:37:55 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:37:55 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:37:55 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:37:55 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:37:55 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:37:55 UPR_STREAMREQ for vb 34 successful
DcpNozzle 2017-04-25T13:37:55.959-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:37:55.959-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
2017/04/25 13:37:56 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 239 successful
CapiNozzle 2017-04-25T13:37:56.469-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:37:56.477-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:37:56.478-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:37:56.478-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:37:56.479-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:dial tcp 10.202.98.33:9091: i/o timeout], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:37:56.487-05:00 [INFO] Send batch count=500
2017/04/25 13:37:56 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:37:56 UPR_STREAMREQ for vb 37 successful
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x6b324d]

goroutine 3734 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).tcpProxy(0xc3bf152fc0, 0x12c, 0xc348926000, 0xc3286f7e30, 0xc27322ae40, 0xc27322aea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:908 +0x7cd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:831 +0xc91

goroutine 1 [chan receive, 4 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 4 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc2080d4370)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080c4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080da001, 0x5ff, 0x5ff, 0xc2080d8080, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080da001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080d0000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d0000, 0x8f07a0, 0xc2080664a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc208066480, 0xc2080d65e0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bc00, 0x7fec37990c10, 0xc208066480, 0x0, 0x0, 0xc2080d65e0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bc00, 0x7fec37990c10, 0xc208066480, 0xc208086280, 0xc20801f660, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bc00, 0x7fec37990c10, 0xc208066480)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b3f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b3f90, 0xc2080be480, 0x7fec3798f398, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 22 [syscall, 4 minutes]:
syscall.Syscall(0x0, 0x0, 0xc20800f000, 0x1000, 0xc20803b1d0, 0xc20814e1e0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc20800f000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc208150f50, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208066b40)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208066b40, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 19 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc20808a0b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 20 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc20808a0b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 33 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc2080e0c30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e0c30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e0bd0, 0xc20812f000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc20802bc70)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081020c0, 0xc20812f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc2081020c0, 0xc20808a268, 0xc20812f000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080e7d80, 0xc20812f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208104a80)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208104a80, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208104a80, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080dcae0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080dcae0, 0xc20816a600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080d8ec0, 0xc20816a600, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080d8ec0, 0xc20816a600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080d8f00, 0xc20816a600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc20810a180, 0xc2080d8c80, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc20810a180, 0x80f160, 0xc2080d8fc0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 3708 [runnable]:
net.(*pollDesc).Wait(0xc2542bba30, 0x77, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitWrite(0xc2542bba30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:93 +0x43
net.(*netFD).Write(0xc2542bb9d0, 0xc324eec000, 0x1000, 0x1000, 0xf8, 0x7fec3798edf0, 0xc27cc78670)
	/usr/local/go/src/net/fd_unix.go:335 +0x5ee
net.(*conn).Write(0xc3d1f575b8, 0xc324eec000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:129 +0xdc
bufio.(*Writer).flush(0xc25aa64980, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:530 +0xe0
bufio.(*Writer).ReadFrom(0xc25aa64980, 0x7fec37990fd0, 0xc27c90e7e0, 0x4ef9, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:654 +0x117
io.Copy(0x7fec37990a58, 0xc25aa64980, 0x7fec37990fd0, 0xc27c90e7e0, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:358 +0x13d
net/http.(*transferWriter).WriteBody(0xc30926fb20, 0x7fec37990a58, 0xc25aa64980, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:214 +0x7c7
net/http.(*Request).write(0xc2742c00d0, 0x7fec37990a58, 0xc25aa64980, 0xc26fa25000, 0xc26d34d3b0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:448 +0xada
net/http.(*persistConn).writeLoop(0xc257d8dad0)
	/usr/local/go/src/net/http/transport.go:951 +0x28d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 23 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc2080842a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 24 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080842b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 25 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 26 [select, 4 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 27 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc208080500, 0x11, 0xc20805ffc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc208080500, 0x11, 0xc20805ffc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc208080500, 0x11, 0xc20805ffc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc208150910)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 28 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20802b6b0, 0xf, 0xc20805efc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20802b6b0, 0xf, 0xc20805efc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20802b6b0, 0xf, 0xc20805efc0, 0xc208066600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc208150aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 29 [select]:
net/http.(*persistConn).roundTrip(0xc271223ef0, 0xc2dcfc3300, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc268468000, 0xc26e76caa0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc268468000, 0x7fec3798efe8, 0xe94600, 0x1e, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc268468000, 0x1e, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc268468000, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc268468000, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc268468000, 0x7fec37990f78, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3369bbb90, 0x9bbca0, 0x6, 0x7fec37990f78, 0xc26e76c8e0, 0x80f820, 0xc32f864318, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc32f864300, 0x9bbca0, 0x6, 0x80f820, 0xc32f864318, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc259101340, 0x33, 0x7fec37990f78, 0xc26e76c8e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/go-couchbase.Connect(0xc259101340, 0x33, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:632 +0xa9
github.com/couchbase/goxdcr/utils.RemotePool(0xc2080d6a60, 0x11, 0xc2080d4680, 0xd, 0xc2080d4690, 0xc, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:94 +0x287
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).cacheRef(0xc208066360, 0xc39dac7dd0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:512 +0x1d3
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).refresh(0xc208066360, 0xc39dac7dd0, 0xc2080800c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:551 +0x154
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).RemoteClusterByUuid(0xc208066360, 0xc2080800c0, 0x20, 0x1, 0xc26f0934a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:178 +0x2a1
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080e02a0, 0xc3aafc7e80, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:480 +0x51d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080e02a0, 0xc3aafc7e80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc208010690)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 30 [select]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208066ae0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:102 +0x75b
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 15 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 16 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 31 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208066af8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 32 [IO wait]:
net.(*pollDesc).Wait(0xc208010840, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010840, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080107e0, 0x0, 0x7fec3798edf0, 0xc23db8f678)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038070, 0xc208169e38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208038070, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208066d20, 0x7fec379925a0, 0xc208038070, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 34 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc2080e0450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e0450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e03f0, 0xc208108000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc20802bb40)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208102050, 0xc208108000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc208102050, 0xc20808a108, 0xc208108000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080d68c0, 0xc208108000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208104360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208104360, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208104360, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080dcc00)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080dcc00, 0xc20816a000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080d9000, 0xc20816a000, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080d9000, 0xc20816a000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080d9040, 0xc20816a000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc20810a480, 0xc2080d8ac0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc20810a480, 0x80f160, 0xc2080d9100, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 234 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x1, 0xc211bb1a00, 0xea, 0x100, 0xc2192ca000, 0xc7, 0x100, 0xc219380120, 0xc219380060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 284 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x2, 0xc219b8c000, 0xea, 0x100, 0xc21a40a800, 0xcf, 0x100, 0xc21c86c7e0, 0xc21c86c720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3682 [select]:
net/http.(*persistConn).writeLoop(0xc337b3a000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 61 [IO wait]:
net.(*pollDesc).Wait(0xc2080e1480, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e1480, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e1420, 0xc208078000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc3cfcd5788)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208102168, 0xc208078000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc208102168, 0xc2080763c8, 0xc208078000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080e7fa0, 0xc208078000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080679e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080679e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076370)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 62 [select]:
net/http.(*persistConn).writeLoop(0xc208076370)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 46 [IO wait]:
net.(*pollDesc).Wait(0xc208010a70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010a70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010a10, 0xc2081be000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc211bcdf48)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038138, 0xc2081be000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc208038138, 0xc208076058, 0xc2081be000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081da760, 0xc2081be000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081044e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081044e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208076000)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 47 [select]:
net/http.(*persistConn).writeLoop(0xc208076000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3308 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15d8b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 667 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0xd0, 0xc20802f000, 0xec, 0x100, 0xc26c8ca800, 0xb2, 0x100, 0xc26d4bf380, 0xc26d4bf2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 594 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x0, 0xc24c522f00, 0xea, 0x100, 0xc24c7a8000, 0xb8, 0x100, 0xc24c870900, 0xc24c870840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3343 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3bef97200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3312 [select]:
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).checkpointing(0xc3bf4285a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:636 +0x419
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:196 +0x292

goroutine 3303 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15d130)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3344 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc3bef97200, 0xc3c7c99950)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 726 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x1, 0xc273719200, 0xea, 0x100, 0xc273635000, 0xc7, 0x100, 0xc2739b14a0, 0xc2739b13e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2013 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316880fc0, 0x0, 0xc20802e600, 0xea, 0x100, 0xc2aee96800, 0xb8, 0x100, 0xc24dd2ec00, 0xc24dd2eb40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1281 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5200, 0x1, 0xc2cc76e600, 0xea, 0x100, 0xc2cc49f800, 0xc7, 0x100, 0xc2cc7acf00, 0xc2cc7ace40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 975 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0xc9, 0xc273200a00, 0xec, 0x100, 0xc26a793000, 0xbd, 0x100, 0xc2687a7b00, 0xc2687a7a40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1641 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x1, 0xc211bb1500, 0xea, 0x100, 0xc2d9579800, 0xc7, 0x100, 0xc2d9777260, 0xc2d97771a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3367 [runnable]:
net/http.(*persistConn).roundTrip(0xc337b3a000, 0xc2dc93efc0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc2786b0000, 0xc27c8533b0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc2786b0000, 0x7fec3798efe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc2786b0000, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc2786b0000, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc2786b0000, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc2786b0000, 0x0, 0x818220, 0xc211be2bb0, 0xeb0d80, 0xc20802b4c0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3bf5fcab7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d4680, 0xd, 0xc2080d4690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b500, 0x9ed550, 0xe, 0xc30d90b260, 0xeb0d80, 0x5, 0x66504c, 0x30000c211be2ba8, 0xc3bf511e40, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b500, 0xc3bf241650, 0xc26ddb23a0, 0xc3d20b5e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3bf4285a0, 0x3, 0xc3c072c518, 0x29, 0x34, 0xc3bf5fcbd0, 0xc3bf2e0300, 0xc3c7dfcf00, 0xc3c832a920, 0xc3c827ec00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 1078 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22240, 0x4, 0xc2a3893500, 0xea, 0x100, 0xc2a38f3800, 0xa2, 0x100, 0xc2a3c931a0, 0xc2a3c930e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 596 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0xc8, 0xc24d112400, 0xec, 0x100, 0xc24cfab800, 0xba, 0x100, 0xc24d2d8e40, 0xc24d2d8d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1645 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0x12e, 0xc2e2974800, 0xec, 0x100, 0xc2e24c1800, 0xb3, 0x100, 0xc2e2990e40, 0xc2e2990d80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 355 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x4, 0xc2321dd100, 0xea, 0x100, 0xc23677e800, 0xa2, 0x100, 0xc2371410e0, 0xc237141020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 661 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x67, 0xc2321ddd00, 0xec, 0x100, 0xc25db63800, 0xb3, 0x100, 0xc25e1b45a0, 0xc25e1b44e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 347 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x130, 0xc21858b200, 0xec, 0x100, 0xc22ff7b800, 0xbc, 0x100, 0xc230d03620, 0xc230d03560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 633 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0x132, 0xc254952c00, 0xec, 0x100, 0xc25b322000, 0xbf, 0x100, 0xc250d7c240, 0xc250d7c180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3728 [runnable]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066cc0, 0x7fec37992650, 0xc232621ae0, 0xc26a578000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038060, 0x7fec37992650, 0xc232621ae0, 0xc26a578000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fec37992650, 0xc232621ae0, 0xc26a578000)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066d20, 0x7fec37992650, 0xc232621ae0, 0xc26a578000)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3d7c095e0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3707 [IO wait]:
net.(*pollDesc).Wait(0xc2542bba30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2542bba30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2542bb9d0, 0xc324d2b000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc27cc78598)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3d1f575b8, 0xc324d2b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc3d1f575b8, 0xc257d8db28, 0xc324d2b000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc25be3fbc0, 0xc324d2b000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2de3dcf00)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2de3dcf00, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc257d8dad0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1312 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5200, 0x66, 0xc2b6246700, 0xec, 0x100, 0xc2c6581000, 0xc8, 0x100, 0xc26fb1f9e0, 0xc26fb1f920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1696 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0x12d, 0xc3063e3500, 0xec, 0x100, 0xc30b4e4800, 0xb2, 0x100, 0xc30bc43f20, 0xc30bc43e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1747 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x0, 0xc314b09500, 0xea, 0x100, 0xc31492e800, 0xb8, 0x100, 0xc314d0d320, 0xc314d0d260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 319 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12e, 0xc219b8d200, 0xec, 0x100, 0xc2122b6800, 0xb3, 0x100, 0xc21c8ffe60, 0xc21c8ffda0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 134 [select]:
net/http.(*persistConn).writeLoop(0xc20808a420)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 133 [runnable]:
net/http.(*persistConn).readLoop(0xc20808a420)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2393 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x1, 0xc24d112100, 0xea, 0x100, 0xc34d4d3800, 0xc7, 0x100, 0xc34d5e0ba0, 0xc34d5e0ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1660 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x5, 0xc2e457e000, 0xea, 0x100, 0xc2e3f6d800, 0xa2, 0x100, 0xc2e4524840, 0xc2e4524780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3342 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3c7dfc660, 0xc3c8017560, 0xc3c80175c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 1222 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5200, 0x2, 0xc2af43c100, 0xea, 0x100, 0xc2aef99800, 0x61, 0x80, 0xc2af3cad20, 0xc2af3cac60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1914 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316881200, 0xc9, 0xc31fd0e700, 0xec, 0x100, 0xc31fd7a400, 0x5d, 0x80, 0xc31fdb93e0, 0xc31fdb9320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3341 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3bf152fc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1259 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5440, 0x12c, 0xc2b6247c00, 0xec, 0x100, 0xc2b8ae9800, 0xc4, 0x100, 0xc2b97d1a40, 0xc2b97d1980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3339 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc3bf152fc0, 0xc3bf2404d0, 0xc3bf153160)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 115 [IO wait]:
net.(*pollDesc).Wait(0xc20c5b60d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c5b60d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c5b6070, 0xc20c5bc000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc21214d478)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208102148, 0xc20c5bc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc208102148, 0xc20808a3c8, 0xc20c5bc000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c58df20, 0xc20c5bc000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2084a0780)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2084a0780, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a370)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 116 [select]:
net/http.(*persistConn).writeLoop(0xc20808a370)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 145 [syscall, 3 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 3602 [chan receive]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc3d6dcf380, 0x9cac80, 0x3, 0xc20802abe8)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc3bed0da80, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc3bed0da80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3d123c000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1344 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5200, 0x0, 0xc2a3892d00, 0xea, 0x100, 0xc2b5da4800, 0xb8, 0x100, 0xc2b4165ce0, 0xc2b4165c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1704 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0xca, 0xc2f973b800, 0xec, 0x100, 0xc2fc07d000, 0xbc, 0x100, 0xc2fd165260, 0xc2fd1651a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2349 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0xcf, 0xc2a410ee00, 0xec, 0x100, 0xc306a6b800, 0xab, 0x100, 0xc2b2878240, 0xc2b2878180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1762 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0xcd, 0xc316282600, 0xec, 0x100, 0xc315ca5800, 0xba, 0x100, 0xc3163d9aa0, 0xc3163d99e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2351 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x2, 0xc2a410f900, 0xea, 0x100, 0xc282272000, 0xa5, 0x100, 0xc29f61bf80, 0xc29f61bec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3362 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3bef97200, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:367 +0x2392
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 696 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x3, 0xc2716b9000, 0xea, 0x100, 0xc270bd0800, 0xb5, 0x100, 0xc271b19a40, 0xc271b19980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 274 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x65, 0xc219b8d400, 0xec, 0x100, 0xc21a5db800, 0xc6, 0x100, 0xc21843d7a0, 0xc21843d6e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 686 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0xd1, 0xc2713f2100, 0xec, 0x100, 0xc2713b2000, 0xb6, 0x100, 0xc27142a720, 0xc27142a660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 671 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x69, 0xc25a6d9b00, 0xec, 0x100, 0xc26f4b7000, 0xba, 0x100, 0xc26fb1ef00, 0xc26fb1ee40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1035 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22240, 0x3, 0xc219b8c500, 0xea, 0x100, 0xc29e5b1000, 0xb5, 0x100, 0xc29f21c5a0, 0xc29f21c4e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 315 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x5, 0xc229d10600, 0xea, 0x100, 0xc229ddc000, 0xbc, 0x100, 0xc229e2d3e0, 0xc229e2d320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1029 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22240, 0x64, 0xc238b01f00, 0xec, 0x100, 0xc292fcb800, 0xc9, 0x100, 0xc29355d5c0, 0xc29355d500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1023 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0xc8, 0xc254953f00, 0xec, 0x100, 0xc26dd75800, 0xbb, 0x100, 0xc27e9ef260, 0xc27e9ef1a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2415 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0x12c, 0xc21c55b100, 0xec, 0x100, 0xc361308000, 0xc4, 0x100, 0xc3613b84e0, 0xc3613b8420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1727 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0xc9, 0xc31458e000, 0xec, 0x100, 0xc314135000, 0xbd, 0x100, 0xc3145455c0, 0xc314545500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3365 [select]:
net/http.(*persistConn).roundTrip(0xc257d8da20, 0xc3488ea4b0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc2682b8d00, 0xc26db55950, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc2682b8d00, 0x7fec3798efe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc2682b8d00, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc2682b8d00, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc2682b8d00, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc2682b8d00, 0x0, 0x818220, 0xc211be2b68, 0xeb0d80, 0xc20802b4c0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3bf5fcab7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d4680, 0xd, 0xc2080d4690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b500, 0x9ed550, 0xe, 0xc323476780, 0xeb0d80, 0x5, 0x66504c, 0x30000c211be2b60, 0xc3bf50cbc0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b500, 0xc3bf241650, 0xc272a6b700, 0xc32e703e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3bf4285a0, 0x1, 0xc3c072c388, 0x64, 0xfc, 0xc3bf5fcbd0, 0xc3bf2e0300, 0xc3c7dfcf00, 0xc3c832a920, 0xc3c827ec00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3681 [select]:
net/http.(*persistConn).readLoop(0xc337b3a000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 353 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12d, 0xc23e20ac00, 0xec, 0x100, 0xc23dcd5800, 0xcf, 0x100, 0xc23e852960, 0xc23e8528a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 295 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12f, 0xc21c55a300, 0xec, 0x100, 0xc21c3e2000, 0xce, 0x100, 0xc21c5beba0, 0xc21c5beae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 267 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xcb, 0xc219b8cd00, 0xec, 0x100, 0xc219665000, 0xc9, 0x100, 0xc219cc69c0, 0xc219cc6900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 305 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xc9, 0xc21c55b200, 0xec, 0x100, 0xc223779000, 0xbd, 0x100, 0xc223b7a6c0, 0xc223b7a600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3347 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc3bef96f00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 339 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x66, 0xc20802ed00, 0xec, 0x100, 0xc229d7c800, 0xc8, 0x100, 0xc21b165e60, 0xc21b165da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2359 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0xc8, 0xc283d13c00, 0xec, 0x100, 0xc323732800, 0xbb, 0x100, 0xc2fc826840, 0xc2fc826780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1949 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316880fc0, 0x66, 0xc325615900, 0xec, 0x100, 0xc326424000, 0xc8, 0x100, 0xc3264cf320, 0xc3264cf260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 623 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0x12c, 0xc25a6d8300, 0xec, 0x100, 0xc259dc4800, 0xa2, 0x100, 0xc25a64e5a0, 0xc25a64e4e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 981 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22240, 0x65, 0xc284229900, 0xec, 0x100, 0xc28a2cb800, 0x9a, 0x100, 0xc28a6aeae0, 0xc28a6aea20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3340 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3bf152fc0, 0xa9012c, 0xc3bf559b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:836 +0x14d1
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3bf152fc0, 0xc39e0e012c, 0xc3bf559b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3bf152fc0, 0xc3c5577340, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3bf152fc0, 0xc3c5577340, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3bf152fc0, 0xc3bf2403f0, 0xc3bf153160, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 3668 [chan receive]:
github.com/couchbase/go-couchbase.(*Bucket).GetStats(0xc336ae8b60, 0x9cac80, 0x3, 0xc20802abe8)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:169 +0x180
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).getDcpStats(0xc3bed0da80, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:263 +0x19e
github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).checkPipelineHealth(0xc3bed0da80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:236 +0x1ef
github.com/couchbase/goxdcr/pipeline_svc.*PipelineSupervisor.(github.com/couchbase/goxdcr/pipeline_svc.checkPipelineHealth)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:143 +0x39
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3d88407e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 642 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x66, 0xc24d113b00, 0xec, 0x100, 0xc259ed6000, 0xc8, 0x100, 0xc23b5d9620, 0xc23b5d9560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1708 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x3, 0xc2fd9ad800, 0xea, 0x100, 0xc2ffbf3800, 0xb5, 0x100, 0xc3003d9920, 0xc3003d9860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3203 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3943d1b00, 0x12c, 0xc3a6f71c00, 0xec, 0x100, 0xc3b1925000, 0xbb, 0x100, 0xc3b2e4df20, 0xc3b2e4de60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3293 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3bf152d80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3299 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15de00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 1320 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5200, 0x65, 0xc254953900, 0xec, 0x100, 0xc298f8d000, 0xc6, 0x100, 0xc2cd7c91a0, 0xc2cd7c90e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3705 [select]:
net/http.(*persistConn).writeLoop(0xc257d8da20)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3485 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3bf152fc0, 0x12c, 0xc3cbc90a00, 0xec, 0x100, 0xc3ca5d6000, 0x7f, 0x80, 0xc3cc0ba0c0, 0xc3cc0ba000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2786 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f42dd40, 0x0, 0xc390921b00, 0xea, 0x100, 0xc390d12000, 0xb8, 0x100, 0xc390d57800, 0xc390d57740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1718 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x66, 0xc30e6f2900, 0xec, 0x100, 0xc30e7ea000, 0xc8, 0x100, 0xc30e990000, 0xc30e805f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2057 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316880fc0, 0x64, 0xc32af31a00, 0xec, 0x100, 0xc32b736800, 0xc9, 0x100, 0xc2c17db8c0, 0xc2c17db800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 641 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0xcb, 0xc2321dc300, 0xec, 0x100, 0xc238845800, 0xc9, 0x100, 0xc25d143740, 0xc25d143680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 707 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3d40, 0x133, 0xc273200300, 0xec, 0x100, 0xc272ecd800, 0xb8, 0x100, 0xc2732971a0, 0xc2732970e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 600 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2171a3b00, 0x65, 0xc254953c00, 0xec, 0x100, 0xc258187800, 0xc6, 0x100, 0xc2593a6ea0, 0xc2593a6de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1043 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0x12d, 0xc249087b00, 0xec, 0x100, 0xc294170800, 0xcf, 0x100, 0xc294eda0c0, 0xc294eda000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1082 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0x132, 0xc2a410f400, 0xec, 0x100, 0xc2a3fd9800, 0xbf, 0x100, 0xc2a43af0e0, 0xc2a43af020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3239 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3943d1b00, 0x12c, 0xc3b4dc7700, 0xec, 0x100, 0xc3bcb55800, 0xbb, 0x100, 0xc3be8a59e0, 0xc3be8a5920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3120 [IO wait]:
net.(*pollDesc).Wait(0xc3a60520d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3a60520d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3a6052070, 0xc3a6444000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc20807c4c0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39f26b460, 0xc3a6444000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc39f26b460, 0xc3a5fe2c08, 0xc3a6444000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3a62d2660, 0xc3a6444000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3a5f63860)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3a5f63860, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3a5fe2bb0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3307 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15d6d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3121 [select]:
net/http.(*persistConn).writeLoop(0xc3a5fe2bb0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3294 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3c3845590, 0xc3c3a36300, 0xc3c3a36360)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 3301 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15da90)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 1017 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22240, 0x0, 0xc254952200, 0xea, 0x100, 0xc25db41800, 0xb8, 0x100, 0xc26f0ca8a0, 0xc26f0ca7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3115 [IO wait]:
net.(*pollDesc).Wait(0xc3a53eac30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3a53eac30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3a53eabd0, 0xc3a6058000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc23630e990)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3a5b42070, 0xc3a6058000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc3a5b42070, 0xc3a4ccc9f8, 0xc3a6058000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3a53e9920, 0xc3a6058000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3a5f63200)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3a5f63200, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3a4ccc9a0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3119 [select]:
net/http.(*persistConn).writeLoop(0xc3a5fe2c60)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2084 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316881200, 0x133, 0xc316282c00, 0xec, 0x100, 0xc30fca2000, 0xb8, 0x100, 0xc3044dfa40, 0xc3044df980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 971 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22240, 0x1, 0xc27d50f400, 0xea, 0x100, 0xc27e426000, 0xc7, 0x100, 0xc287006d80, 0xc287006cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3292 [select]:
net/http.(*persistConn).roundTrip(0xc257d8dad0, 0xc34878ff80, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc2742c00d0, 0xc259101c80, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc2742c00d0, 0x7fec3798efe8, 0xc20802c090, 0x32, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc2742c00d0, 0x32, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc2742c00d0, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc2742c00d0, 0xeb0d80, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc2742c00d0, 0x29e8d60800, 0x812be0, 0xc34878ff20, 0xeb0d80, 0xc3bf28dcb0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc3bf2a1350, 0x18, 0xc27c90e180, 0x1a, 0xc27c90e101, 0xc2080d4680, 0xd, 0xc2080d4690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:173 +0x201
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchGetMeta(0xc3bf152d80, 0xa60064, 0xc3c3805050, 0xc34896bce8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:499 +0x948
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3bf152d80, 0xc3c3808150, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:452 +0x34f
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3bf152d80, 0xc3bf2402a0, 0xc3bf152f20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 1039 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0x12e, 0xc211bb1100, 0xec, 0x100, 0xc2a1f02800, 0xb3, 0x100, 0xc2a2792120, 0xc2a2792060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2065 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316880fc0, 0x1, 0xc31fa45f00, 0xea, 0x100, 0xc311bca000, 0xc7, 0x100, 0xc307f1ee40, 0xc307f1ed80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3296 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc3bef96f00, 0xc3c3805e60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 885 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0x12c, 0xc27d50f900, 0xec, 0x100, 0xc27dbb1800, 0x65, 0x80, 0xc27de83da0, 0xc27de83ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1629 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0xc8, 0xc211e05900, 0xec, 0x100, 0xc2d8aca000, 0xbb, 0x100, 0xc2d8af3680, 0xc2d8af35c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3346 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3bef96f00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:367 +0x2392
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 967 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc273e22480, 0x12c, 0xc283d13900, 0xec, 0x100, 0xc2889f9000, 0xa2, 0x100, 0xc289298120, 0xc289298060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3276 [semacquire]:
sync.(*WaitGroup).Wait(0xc3c832a920)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc3bf4285a0, 0xc20800a700, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc20814e140, 0x7fec37998de0, 0xc3bf1653f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7fec37998de0, 0xc3bf1653f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 2409 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x69, 0xc2321dd300, 0xec, 0x100, 0xc3599a7000, 0xba, 0x100, 0xc35a518480, 0xc35a5183c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3738 [runnable]:
net/http.func·019()
	/usr/local/go/src/net/http/transport.go:519
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:522 +0x335

goroutine 3684 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc2375b6e10, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x221
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2375b6e10, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc3d6dcf380, 0x1, 0x9cac80, 0x3, 0xc2b72952c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 2061 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316881200, 0x12c, 0xc325615600, 0xec, 0x100, 0xc328f2b000, 0xc4, 0x100, 0xc2d82ea780, 0xc2d82ea6c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1332 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5440, 0x12e, 0xc2b9c9a100, 0xec, 0x100, 0xc2c714d800, 0xa3, 0x100, 0xc27322a000, 0xc270fddf20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3733 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xeb1a20, 0xc26df215f0, 0xc26df216b0, 0xc208010380, 0xc3286f7a40, 0xc2080e7c20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:269 +0xa90
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 1314 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5440, 0x12d, 0xc254952700, 0xec, 0x100, 0xc2b15ef800, 0xcf, 0x100, 0xc2b3a9efc0, 0xc2b3a9ef00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3304 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf3c8000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 1285 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5440, 0xc8, 0xc2ca56ab00, 0xec, 0x100, 0xc2ca08d800, 0xbb, 0x100, 0xc2ca77c4e0, 0xc2ca77c420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2471 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0xcc, 0xc36e223f00, 0xec, 0x100, 0xc36e7e6800, 0xb7, 0x100, 0xc36ec2ade0, 0xc36ec2ad20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1263 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5200, 0x64, 0xc2b9c9b900, 0xec, 0x100, 0xc2bac31800, 0xc9, 0x100, 0xc2bb39e9c0, 0xc2bb39e900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1527 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x5, 0xc24c523500, 0xea, 0x100, 0xc2937f0400, 0x46, 0x80, 0xc2952dbda0, 0xc2952dbce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1353 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a52f5440, 0xc9, 0xc2a410e000, 0xec, 0x100, 0xc2b71c5800, 0xbd, 0x100, 0xc2806eb620, 0xc2806eb560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3118 [IO wait]:
net.(*pollDesc).Wait(0xc3a6052140, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3a6052140, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3a60520e0, 0xc3a6442000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc3a6042fc8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39f26b458, 0xc3a6442000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc39f26b458, 0xc3a5fe2cb8, 0xc3a6442000, 0x1000, 0x1000, 0x7fec3797d010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3a62d2600, 0xc3a6442000, 0x1000, 0x1000, 0xc1ee8d94b9, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3a5f63800)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3a5f63800, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3a5fe2c60)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3363 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc3bef97200, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 3364 [select]:
net/http.(*Transport).getConn(0xc20802c090, 0xc2741f8820, 0x0, 0xc26d195560, 0x4, 0xc26d195567, 0x11, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:525 +0x608
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc2741f8820, 0xc26d1958c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:228 +0x4d4
net/http.send(0xc2741f8820, 0x7fec3798efe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc2741f8820, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc2741f8820, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc2741f8820, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc2741f8820, 0x0, 0x818220, 0xc23bdedb58, 0xeb0d80, 0xc20802b4c0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3bf5fcab7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d4680, 0xd, 0xc2080d4690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b500, 0x9ed550, 0xe, 0xc3486d2720, 0xeb0d80, 0x5, 0x66504c, 0x30000c23bdedb50, 0xc3bf2e1440, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b500, 0xc3bf241650, 0xc27c904160, 0xc2c3611e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3bf4285a0, 0x0, 0xc3c072c2c0, 0x64, 0x160, 0xc3bf5fcbd0, 0xc3bf2e0300, 0xc3c7dfcf00, 0xc3c832a920, 0xc3c827ec00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3218 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3943d18c0, 0x64, 0xc3af314c00, 0xec, 0x100, 0xc3b2816800, 0xac, 0x100, 0xc3b2ef38c0, 0xc3b2ef3800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3310 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3bec6b5f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 3345 [runnable]:
io.ReadFull(0x7fec37990b08, 0xc3bcb018b0, 0xc345616000, 0x385, 0x385, 0x385, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc39401ee78, 0x7fec37990b08, 0xc3bcb018b0, 0xc3c80bf660, 0x18, 0x18, 0xc2dc70801f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3bf591560, 0xc3c8017aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 2798 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f42dd40, 0x0, 0xc392ca0700, 0xea, 0x100, 0xc392f3f800, 0xb8, 0x100, 0xc393800960, 0xc3938008a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2047 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316881200, 0x12d, 0xc2fd9ad700, 0xec, 0x100, 0xc3131ab000, 0xcf, 0x100, 0xc3329ba1e0, 0xc332a5f320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3736 [IO wait]:
net.(*pollDesc).Wait(0xc3369bbe20, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3369bbe20, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3369bbdc0, 0xc273e0b000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc27d01d3e0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc23bdedb28, 0xc273e0b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc23bdedb28, 0xc271223f48, 0xc273e0b000, 0x1000, 0x1000, 0x7fec3797d010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc27c91e580, 0xc273e0b000, 0x1000, 0x1000, 0xc204c2124b, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc305008060)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc305008060, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc271223ef0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2447 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x64, 0xc36daa6700, 0xec, 0x100, 0xc36db26000, 0xc9, 0x100, 0xc36db6d8c0, 0xc36db6d800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2403 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0xca, 0xc24c522700, 0xec, 0x100, 0xc3529b3800, 0xbc, 0x100, 0xc352ee4ba0, 0xc352ee4ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3116 [select]:
net/http.(*persistConn).writeLoop(0xc3a4ccc9a0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2435 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x3, 0xc219b8d900, 0xea, 0x100, 0xc36558e800, 0xb5, 0x100, 0xc36724c600, 0xc36724c540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1571 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0x12d, 0xc2713f3f00, 0xec, 0x100, 0xc2c10eac00, 0x2f, 0x40, 0xc2be456cc0, 0xc2be456c00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3309 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc3bec6b5e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 2043 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316880fc0, 0x65, 0xc2fd9ac700, 0xec, 0x100, 0xc32d4bc800, 0xba, 0x100, 0xc3305b2540, 0xc3305b2480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1676 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301440, 0x64, 0xc2ed9df700, 0xec, 0x100, 0xc2ef372000, 0xc9, 0x100, 0xc2ef49c0c0, 0xc2ef49c000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1672 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc296301680, 0x12c, 0xc2ed9dec00, 0xec, 0x100, 0xc2edd06000, 0xc4, 0x100, 0xc2eddc0480, 0xc2eddc03c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2389 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0xce, 0xc2713f3100, 0xec, 0x100, 0xc34716b000, 0xab, 0x100, 0xc3476de1e0, 0xc3476de120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3291 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc3bf152d80, 0xc3bf240380, 0xc3bf152f20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 2433 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9440, 0x12f, 0xc36c8afa00, 0xec, 0x100, 0xc36c671000, 0xce, 0x100, 0xc36caa1da0, 0xc36caa1ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3297 [runnable]:
io.ReadFull(0x7fec37990b08, 0xc3bece6c30, 0xc273e0a000, 0xe30, 0xe30, 0xe30, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc2081c2e78, 0x7fec37990b08, 0xc3bece6c30, 0xc3c3b15b80, 0x18, 0x18, 0xc34885f4bf, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3bf1cbcb0, 0xc3c3a371a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3298 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15cf50)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2261 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x2, 0xc2ee6a1e00, 0xea, 0x100, 0xc2f3d76000, 0x6f, 0x80, 0xc2f3d3bf80, 0xc2f3d3bec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2776 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f6cc000, 0xca, 0xc38e682c00, 0xec, 0x100, 0xc38e8b6000, 0xbc, 0x100, 0xc38eb4bbc0, 0xc38eb4bb00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2367 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2f9fd9200, 0x5, 0xc2716b8000, 0xea, 0x100, 0xc341961000, 0xbc, 0x100, 0xc3420705a0, 0xc3420704e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2028 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316881200, 0xc8, 0xc3379ef800, 0xec, 0x100, 0xc2dc8a4000, 0xbb, 0x100, 0xc264b1f620, 0xc264b1f560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2019 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc316881200, 0xc9, 0xc3247e3900, 0xec, 0x100, 0xc32a966000, 0xa4, 0x100, 0xc32a9b31a0, 0xc32a9b30e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3659 [select]:
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc2f8fb8080)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:557 +0x48b
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 3305 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf3c8190)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3281 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15cd70)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3683 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc23686fa70, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x221
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc23686fa70, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc3d6dcf380, 0x0, 0x9cac80, 0x3, 0xc2b72952c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 3746 [runnable]:
net.(*pollDesc).Wait(0xc30926faa0, 0x77, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitWrite(0xc30926faa0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:93 +0x43
net.(*netFD).connect(0xc30926fa40, 0x0, 0x0, 0x7fec3798fa20, 0xc27c90e6a0, 0xed0918ea2, 0x29739ac0, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:114 +0x24b
net.(*netFD).dial(0xc30926fa40, 0x7fec3798f9d0, 0x0, 0x7fec3798f9d0, 0xc26d34d2f0, 0xed0918ea2, 0x29739ac0, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/net/sock_posix.go:139 +0x37a
net.socket(0x9d8aa0, 0x3, 0x2, 0x1, 0x0, 0xc26d34d200, 0x7fec3798f9d0, 0x0, 0x7fec3798f9d0, 0xc26d34d2f0, ...)
	/usr/local/go/src/net/sock_posix.go:91 +0x422
net.internetSocket(0x9d8aa0, 0x3, 0x7fec3798f9d0, 0x0, 0x7fec3798f9d0, 0xc26d34d2f0, 0xed0918ea2, 0x29739ac0, 0xeb0f40, 0x1, ...)
	/usr/local/go/src/net/ipsock_posix.go:137 +0x148
net.dialTCP(0x9d8aa0, 0x3, 0x0, 0xc26d34d2f0, 0xed0918ea2, 0x29739ac0, 0xeb0f40, 0xc27cc78580, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:156 +0x125
net.dialSingle(0x9d8aa0, 0x3, 0xc3bf2a1357, 0x11, 0x0, 0x0, 0x7fec3798f940, 0xc26d34d2f0, 0xed0918ea2, 0x29739ac0, ...)
	/usr/local/go/src/net/dial.go:235 +0x3f5
net.func·016(0xed0918ea2, 0xe29739ac0, 0xeb0f40, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:163 +0x12c
net.dial(0x9d8aa0, 0x3, 0x7fec3798f940, 0xc26d34d2f0, 0xc3d20afb08, 0xed0918ea2, 0xe29739ac0, 0xeb0f40, 0x0, 0x0, ...)
	/usr/local/go/src/net/fd_unix.go:40 +0x6f
net.(*Dialer).Dial(0xc20800a0c0, 0x9d8aa0, 0x3, 0xc3bf2a1357, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:170 +0x4be
net.*Dialer.Dial·fm(0x9d8aa0, 0x3, 0xc3bf2a1357, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:38 +0x79
net/http.(*Transport).dial(0xc20802c090, 0x9d8aa0, 0x3, 0xc3bf2a1357, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:479 +0x84
net/http.(*Transport).dialConn(0xc20802c090, 0x0, 0xc3bf2a1350, 0x4, 0xc3bf2a1357, 0x11, 0x4, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:564 +0x1678
net/http.func·019()
	/usr/local/go/src/net/http/transport.go:520 +0x42
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:522 +0x335

goroutine 3656 [runnable]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066cc0, 0x7fec37992650, 0xc211b7a6e0, 0xc26a7cdad0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038060, 0x7fec37992650, 0xc211b7a6e0, 0xc26a7cdad0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fec37992650, 0xc211b7a6e0, 0xc26a7cdad0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066d20, 0x7fec37992650, 0xc211b7a6e0, 0xc26a7cdad0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3d7c08140)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3700 [select]:
net/http.(*persistConn).writeLoop(0xc22e1528f0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3735 [runnable]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3bf152fc0, 0x12c, 0xc2716b8500, 0xec, 0x100, 0xc260cec000, 0x7f, 0x80, 0xc348926000, 0xc27322ae40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3731 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3bf152d80, 0x0, 0xc2716b8100, 0xea, 0x100, 0xc256314000, 0xb4, 0x100, 0xc27322a5a0, 0xc27322a4e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3295 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3bef96f00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2756 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f42dd40, 0x1, 0xc381674b00, 0xea, 0x100, 0xc380a2c800, 0xc7, 0x100, 0xc3818c05a0, 0xc3818c04e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3300 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15d310)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3737 [runnable]:
net/http.(*persistConn).writeLoop(0xc271223ef0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3685 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc2375b6900, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x221
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2375b6900, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc3d6dcf380, 0x2, 0x9cac80, 0x3, 0xc2b72952c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 3306 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15d4f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3366 [runnable]:
net/http.(*Transport).getConn(0xc20802c090, 0xc2742c0340, 0x0, 0xc26d34cea0, 0x4, 0xc26d34cea7, 0x11, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:525 +0x608
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc2742c0340, 0xc26d34d170, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:228 +0x4d4
net/http.send(0xc2742c0340, 0x7fec3798efe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc2742c0340, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc2742c0340, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc2742c0340, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc2742c0340, 0x0, 0x818220, 0xc211be2c18, 0xeb0d80, 0xc20802b4c0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3bf5fcab7, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d4680, 0xd, 0xc2080d4690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b500, 0x9ed550, 0xe, 0xc307424900, 0xeb0d80, 0x5, 0x66504c, 0x30000c211be2c10, 0xc3bf510600, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b500, 0xc3bf241650, 0xc27c90e480, 0xc32e701e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3bf4285a0, 0x2, 0xc3c072c450, 0x64, 0x98, 0xc3bf5fcbd0, 0xc3bf2e0300, 0xc3c7dfcf00, 0xc3c832a920, 0xc3c827ec00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3302 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3bf15dc20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3704 [runnable]:
net.(*pollDesc).Wait(0xc25f0e7e20, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc25f0e7e20, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc25f0e7dc0, 0xc322830000, 0x1000, 0x1000, 0x0, 0x7fec3798edf0, 0xc3488ea5b8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3d1f575a0, 0xc322830000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fec37990b08, 0xc3d1f575a0, 0xc257d8da78, 0xc322830000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc25c090a60, 0xc322830000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc25fe64420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc25fe64420, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc257d8da20)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3460 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3bf152d80, 0x0, 0xc3c94c3100, 0xea, 0x100, 0xc3c9733800, 0xb4, 0x100, 0xc3cb2c8420, 0xc3cb2c8360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3719 [runnable]:
encoding/binary.bigEndian.Uint64(0xc24aa60d10, 0x8, 0x8, 0x18)
	/usr/local/go/src/encoding/binary/binary.go:109
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc305008000, 0x7fec37990b08, 0xc3d1f56fa8, 0xc24aa60d00, 0x18, 0x18, 0x18, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:169 +0x4ea
github.com/couchbase/gomemcached/client.getResponse(0x7fec37990b08, 0xc3d1f56fa8, 0xc24aa60d00, 0x18, 0x18, 0xc305008000, 0x7d, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc2400f6840, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x25d
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2400f6840, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc336ae8b60, 0x2, 0x9cac80, 0x3, 0xc2f1c46960)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 3718 [runnable]:
net.(*netFD).readUnlock(0xc23ae159d0)
	/usr/local/go/src/net/fd_unix.go:167
net.(*netFD).Read(0xc23ae159d0, 0xc24aa60ec0, 0x18, 0x18, 0x18, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:253 +0x52d
net.(*conn).Read(0xc3d1f56fb0, 0xc24aa60ec0, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7fec37990b08, 0xc3d1f56fb0, 0xc24aa60ec0, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7fec37990b08, 0xc3d1f56fb0, 0xc24aa60ec0, 0x18, 0x18, 0x76, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d
github.com/couchbase/gomemcached.(*MCResponse).Receive(0xc307c6d800, 0x7fec37990b08, 0xc3d1f56fb0, 0xc24aa60ec0, 0x18, 0x18, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_res.go:154 +0x11e
github.com/couchbase/gomemcached/client.getResponse(0x7fec37990b08, 0xc3d1f56fb0, 0xc24aa60ec0, 0x18, 0x18, 0xc307c6d800, 0x8e, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/transport.go:35 +0xe1
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc2400f6a80, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x25d
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2400f6a80, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc336ae8b60, 0x1, 0x9cac80, 0x3, 0xc2f1c46960)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 3717 [runnable]:
github.com/couchbase/gomemcached/client.(*Client).Stats(0xc2541192f0, 0x9cac80, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:547 +0x221
github.com/couchbase/gomemcached/client.(*Client).StatsMap(0xc2541192f0, 0x9cac80, 0x3, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/mc.go:570 +0x88
github.com/couchbase/go-couchbase.getStatsParallel(0xc336ae8b60, 0x0, 0x9cac80, 0x3, 0xc2f1c46960)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:140 +0x220
created by github.com/couchbase/go-couchbase.(*Bucket).GetStats
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/client.go:164 +0x113

goroutine 3748 [chan receive]:
net/http.func·016()
	/usr/local/go/src/net/http/transport.go:507 +0x65
created by net/http.func·017
	/usr/local/go/src/net/http/transport.go:513 +0xba

goroutine 3747 [runnable]:
net.(*pollDesc).Wait(0xc30926fb10, 0x77, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitWrite(0xc30926fb10, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:93 +0x43
net.(*netFD).connect(0xc30926fab0, 0x0, 0x0, 0x7fec3798fa20, 0xc27c90e7c0, 0xed0918ea2, 0x29797a87, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:114 +0x24b
net.(*netFD).dial(0xc30926fab0, 0x7fec3798f9d0, 0x0, 0x7fec3798f9d0, 0xc26d34d410, 0xed0918ea2, 0x29797a87, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/net/sock_posix.go:139 +0x37a
net.socket(0x9d8aa0, 0x3, 0x2, 0x1, 0x0, 0xc26d34d400, 0x7fec3798f9d0, 0x0, 0x7fec3798f9d0, 0xc26d34d410, ...)
	/usr/local/go/src/net/sock_posix.go:91 +0x422
net.internetSocket(0x9d8aa0, 0x3, 0x7fec3798f9d0, 0x0, 0x7fec3798f9d0, 0xc26d34d410, 0xed0918ea2, 0x29797a87, 0xeb0f40, 0x1, ...)
	/usr/local/go/src/net/ipsock_posix.go:137 +0x148
net.dialTCP(0x9d8aa0, 0x3, 0x0, 0xc26d34d410, 0xed0918ea2, 0x29797a87, 0xeb0f40, 0xc27cc78600, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:156 +0x125
net.dialSingle(0x9d8aa0, 0x3, 0xc26d34cea7, 0x11, 0x0, 0x0, 0x7fec3798f940, 0xc26d34d410, 0xed0918ea2, 0x29797a87, ...)
	/usr/local/go/src/net/dial.go:235 +0x3f5
net.func·016(0xed0918ea2, 0xe29797a87, 0xeb0f40, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:163 +0x12c
net.dial(0x9d8aa0, 0x3, 0x7fec3798f940, 0xc26d34d410, 0xc3d20b1b08, 0xed0918ea2, 0xe29797a87, 0xeb0f40, 0x0, 0x0, ...)
	/usr/local/go/src/net/fd_unix.go:40 +0x6f
net.(*Dialer).Dial(0xc20800a0c0, 0x9d8aa0, 0x3, 0xc26d34cea7, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:170 +0x4be
net.*Dialer.Dial·fm(0x9d8aa0, 0x3, 0xc26d34cea7, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:38 +0x79
net/http.(*Transport).dial(0xc20802c090, 0x9d8aa0, 0x3, 0xc26d34cea7, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:479 +0x84
net/http.(*Transport).dialConn(0xc20802c090, 0x0, 0xc26d34cea0, 0x4, 0xc26d34cea7, 0x11, 0xc2bced3020, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:564 +0x1678
net/http.func·019()
	/usr/local/go/src/net/http/transport.go:520 +0x42
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:522 +0x335

goroutine 3699 [runnable]:
net/http.(*persistConn).readLoop(0xc22e1528f0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f
[goport] 2017/04/25 13:37:58 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:37:58.627-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:37:58.628-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:37:58.628-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:37:58.678-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:37:58.685-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:37:58.685-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:37:58.685-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:37:58.685-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:37:58.685-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:37:58.685-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:37:58.685-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:37:58.685-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:37:58.685-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:37:58.685-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:37:58.685-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:37:58.685-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:37:58.685-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:37:58.685-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:37:58.685-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:37:58.685-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:37:58.685-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:37:58.686-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:37:58.686-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:37:58.687-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
ReplicationSpecChangeListener 2017-04-25T13:37:58.695-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:37:58.695-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
RemoteClusterChangeListener 2017-04-25T13:37:58.696-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:37:58.696-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
PipelineManager 2017-04-25T13:37:59.399-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:38:13.738-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:38:13.743-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:38:13.811-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:13.811-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:38:13.811-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:38:13.843-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:13.844-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:38:13.844-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:13.844-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:38:13.844-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:13.844-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:38:13.844-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:38:13.847-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:38:13.848-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
GenericSupervisor 2017-04-25T13:38:13.849-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:38:13.849-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
GenericSupervisor 2017-04-25T13:38:13.849-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:13.849-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:13.849-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:13.849-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:13.849-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:38:13.849-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
CheckpointManager 2017-04-25T13:38:13.849-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
CapiUtils 2017-04-25T13:38:13.870-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:13.870-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:13.870-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:38:13.870-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
StatisticsManager 2017-04-25T13:38:13.871-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:38:13.871-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
XDCRFactory 2017-04-25T13:38:13.871-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:38:13.871-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006 is constructed, start it
GenericPipeline 2017-04-25T13:38:13.872-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[142 126 153 6 20 45 68 96 41 73 108 127 46 81 27 103 137 19 58 116 125 163 107 150 7 69 95 28 37 54 100 115 160 164 167 16 35 70 4 101 140 162 168 44 76 148 110 128 161 8 105 3 65 56 120 143 10 59 131 144 11 13 38 139 77 90 132 158 2 17 26 93 12 71 82 138 169 32 67 84 114 134 1 52 152 57 94 113 104 156 159 36 60 91 23 25 109 136 33 61 92 124 14 34 39 48 9 53 146 166 64 133 63 87 21 89 102 118 157 5 119 22 149 86 129 147 40 80 31 42 83 106 62 78 18 66 165 117 121 123 43 74 111 51 79 98 151 155 88 130 30 75 97 15 29 0 49 55 85 50 112 154 170 72 24 47 141 145 122 99 135] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[293 184 187 263 291 186 230 290 179 328 174 322 329 228 252 191 201 241 250 288 300 302 303 196 229 282 280 188 237 185 216 264 271 244 195 238 192 248 309 278 337 193 204 226 255 274 298 183 281 336 310 311 173 327 275 294 297 320 285 243 272 319 175 209 223 259 222 296 338 269 176 224 225 276 307 206 239 304 295 301 331 334 172 324 190 268 217 318 314 219 246 270 315 321 213 171 203 286 306 316 308 214 227 240 211 245 256 205 236 284 220 233 330 335 180 197 202 305 339 181 260 333 279 332 207 218 249 200 299 182 194 254 262 317 215 177 258 242 312 273 289 189 265 210 231 199 232 261 340 326 323 221 178 313 277 283 325 251 253 267 287 235 208 266 292 198 234 247 257 212]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[214 227 240 211 245 256 205 236 284 220 233 330 335 180 197 202 305 339 181 260 333 279 332 207 218 249 200 299 182 194 254 262 317 215 177 258 242 312 273 289 189 265 210 231 199 232 261 340 326 323 221 178 313 277 283 325 251 253 267 287 235 208 266 292 198 234 247 257 212 293 184 187 263 291 186 230 290 179 328 174 322 329 228 252 191 201 241 250 288 300 302 303 196 229 282 280 188 237 185 216 264 271 244 195 238 192 248 309 278 337 193 204 226 255 274 298 183 281 336 310 311 173 327 275 294 297 320 285 243 272 319 175 209 223 259 222 296 338 269 176 224 225 276 307 206 239 304 295 301 331 334 172 324 190 268 217 318 314 219 246 270 315 321 213 171 203 286 306 316 308] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[9 53 146 166 64 133 63 87 21 89 102 118 157 5 119 22 149 86 129 147 40 80 31 42 83 106 62 78 18 66 165 117 121 123 43 74 111 51 79 98 151 155 88 130 30 75 97 15 29 0 49 55 85 50 112 154 170 72 24 47 141 145 122 99 135 142 126 153 6 20 45 68 96 41 73 108 127 46 81 27 103 137 19 58 116 125 163 107 150 7 69 95 28 37 54 100 115 160 164 167 16 35 70 4 101 140 162 168 44 76 148 110 128 161 8 105 3 65 56 120 143 10 59 131 144 11 13 38 139 77 90 132 158 2 17 26 93 12 71 82 138 169 32 67 84 114 134 1 52 152 57 94 113 104 156 159 36 60 91 23 25 109 136 33 61 92 124 14 34 39 48]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 replication_type:capi active:true worker_batch_size:500 failure_restart_interval:10 log_level:Info target_nozzle_per_node:2 filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:38:13.872-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006...
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:13.876-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
StatisticsManager 2017-04-25T13:38:13.876-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:38:13.877-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:38:13.877-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:38:13.877-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:38:13.877-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:13.877-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:38:13.877-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:38:13.877-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:38:13.877-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:38:13.877-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:38:13.878-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:38:13.878-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:38:13.879-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:38:13.880-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:38:13.934-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:38:13.942-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:13.942-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:13.942-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:38:13.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:38:13.950-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:38:13.995-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:38:13.995-05:00 [INFO] connectionSize=5
TCPConnPoolMgr 2017-04-25T13:38:13.997-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:38:13.997-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:38:13.997-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:38:13.998-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:38:14.015-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:38:14.015-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:38:14.031-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:38:14.113-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:38:14.113-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:14.113-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:38:14.113-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:38:14.113-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:38:14.113-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:38:14.214-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:38:14.214-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:38:14.214-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:38:14.251-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:38:14.251-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:38:14.414-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:38:14.414-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:14.414-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:38:14.414-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:38:14.414-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:38:14.414-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:38:14.414-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006 is started----------
PipelineManager 2017-04-25T13:38:14.414-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:38:14.414-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:38:14.414-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:38:14.525-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:38:14.525-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:38:14.526-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:38:14.526-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:38:14 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 209 successful
CapiNozzle 2017-04-25T13:38:14.846-05:00 [INFO] Send batch count=180
StatisticsManager 2017-04-25T13:38:14.937-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:38:14 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:38:14 UPR_STREAMREQ for vb 117 successful
CapiNozzle 2017-04-25T13:38:14.967-05:00 [INFO] Send batch count=6
2017/04/25 13:38:15 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 225 successful
CapiNozzle 2017-04-25T13:38:15.158-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:15.179-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:38:15.213-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:15.220-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
2017/04/25 13:38:15 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 242 successful
CapiNozzle 2017-04-25T13:38:15.224-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:38:15.242-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
2017/04/25 13:38:15 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 144 successful
CapiNozzle 2017-04-25T13:38:15.300-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:15.311-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
2017/04/25 13:38:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 254 successful
CapiNozzle 2017-04-25T13:38:15.395-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:15.395-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:38:15.395-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:38:15.395-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries]
ReplicationManager 2017-04-25T13:38:15.395-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries]
PipelineManager 2017-04-25T13:38:15.402-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:38:15.403-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:38:15.405-05:00 [INFO] err_list=[{"time":"2017-04-25T13:38:15.405644336-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries"}]
PipelineManager 2017-04-25T13:38:15.405-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:38:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 263 successful
CapiNozzle 2017-04-25T13:38:15.476-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:15.476-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:15.504-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
2017/04/25 13:38:15 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 70 successful
CapiNozzle 2017-04-25T13:38:15.630-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:15.645-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:38:15 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 293 successful
StatisticsManager 2017-04-25T13:38:15.877-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:38:15.878-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006 {"Errors": "[{\"time\":\"2017-04-25T13:38:15.405644336-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 66, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 444, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 66, "set_filtered": 0, "set_received_from_dcp": 444, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries]", "Status": "Pending"}

2017/04/25 13:38:15 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:38:15 UPR_STREAMREQ for vb 191 successful
CheckpointManager 2017-04-25T13:38:15.952-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
CheckpointManager 2017-04-25T13:38:15.952-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:38:15.964-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:15.986-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
2017/04/25 13:38:16 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:38:16 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:38:16 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:38:16 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:38:16 UPR_STREAMREQ for vb 197 successful
DcpNozzle 2017-04-25T13:38:16.135-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:38:16.135-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:38:16.157-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:16.168-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:38:16.510-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:16.523-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:38:16.618-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:16.618-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:38:16.618-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:38:16.618-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:16.618-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:16.689-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:16.689-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:16.711-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:38:16.880-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:16.880-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:38:16.880-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:38:16.880-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:16.881-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:16.964-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:16.965-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:16.997-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:38:17.001-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:17.034-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:38:17.230-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:17.254-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:38:17.466-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:17.533-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:38:17.568-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:17.568-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:38:17.568-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:38:17.568-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:17.569-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:17.654-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:17.654-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:17.676-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:38:18.136-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:18.162-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:38:18.327-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:18.368-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:38:18.777-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:18.777-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:38:18.777-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:38:18.777-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:18.778-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:18.873-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:18.873-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:18.888-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:38:19.052-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:19.077-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:38:19.349-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:19.373-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:38:19.549-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:19.549-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:38:19.549-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:38:19.549-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:19.549-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:19.610-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:19.610-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:19.627-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:38:19.742-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:19.782-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:38:20.009-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:20.028-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:38:20.230-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:20.230-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:38:20.230-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:38:20.230-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:20.230-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:20.499-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:20.532-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:38:20.554-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:20.554-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:20.568-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:38:20.970-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:20.970-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:38:20.970-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:38:20.970-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:20.970-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:21.020-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:21.021-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:21.039-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:38:21.220-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:21.243-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:38:21.403-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:21.415-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:38:21.644-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:21.664-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:38:21.803-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:21.819-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:38:22.021-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:22.021-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:38:22.021-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:38:22.021-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:22.021-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:22.095-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:22.095-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:22.111-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:38:22.205-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:22.205-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:38:22.205-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:38:22.205-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:22.205-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:22.288-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:22.288-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:22.301-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:38:22.474-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:22.487-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:38:22.630-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:22.645-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:38:22.877-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:22.905-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:38:23.069-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:23.088-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:38:23.287-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:23.287-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:38:23.287-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:38:23.287-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:23.287-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:23.375-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:23.375-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:23.390-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:38:23.474-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:23.474-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:38:23.474-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:38:23.474-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:23.474-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:23.577-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:23.577-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:23.590-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:38:23.841-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:38:23.861-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
GenericSupervisor 2017-04-25T13:38:23.886-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:38:23.886-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:38:23.886-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:38:24.013-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:24.027-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
ToplogyChangeDetector 2017-04-25T13:38:24.030-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:38:24.297-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:38:24.313-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:38:24.725-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:24.754-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:38:24.972-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:38:24.972-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:38:24.972-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:38:24.972-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:24.972-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:25.067-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:25.067-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:25.078-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:38:25.147-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:25.147-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:38:25.147-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:38:25.147-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:25.147-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:25.224-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:25.224-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:25.235-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
PipelineManager 2017-04-25T13:38:25.405-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries 
PipelineManager 2017-04-25T13:38:25.406-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:38:25.406-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:38:25.406-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
PipelineRuntimeCtx 2017-04-25T13:38:25.406-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:38:25.406-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:25.406-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:38:25.406-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:25.4062607 -0500 CDT
GenericSupervisor 2017-04-25T13:38:25.406-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:25.406-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:38:25.406-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:38:25.406-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:38:25.406-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:38:25.406-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:38:25.406-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:38:25.406-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:38:25.406-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:38:25.406-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-844671006 has been stopped
GenericPipeline 2017-04-25T13:38:25.406-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-844671006 is stopped
PipelineManager 2017-04-25T13:38:25.406-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:38:25.406-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:15.405644336-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:38:25.406-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:38:25.407-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
GenericPipeline 2017-04-25T13:38:25.407-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
GenericPipeline 2017-04-25T13:38:25.407-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
GenericPipeline 2017-04-25T13:38:25.407-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-844671006
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:25.407-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:38:25.407-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:25.407-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:38:25.407-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:38:25.407-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:25.407-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:38:25.407-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:25.408-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:38:25.408-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:25.408-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:38:25.408-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:25.408-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:25.408-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:25.407881549 -0500 CDT
DcpNozzle 2017-04-25T13:38:25.408-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:38:25.408-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:25.408241584 -0500 CDT
DcpNozzle 2017-04-25T13:38:25.408-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:38:25.409-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:38:25.409-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:38:25.409-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:38:25.409-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:38:25.409-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:38:25.409-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:25.410-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:25.410-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:38:25 Stream Closed for vb 0 StreamEnd simulated
GenericPipeline 2017-04-25T13:38:25.410-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:25.410-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:38:25.415-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:38:25.415-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:25.41567293 -0500 CDT
DcpNozzle 2017-04-25T13:38:25.415-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:38:25.415-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:38:25.415-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:25.415777756 -0500 CDT
GenericPipeline 2017-04-25T13:38:25.415-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:25.415-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:38:25.415-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:38:25.415-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:25.415-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:38:25 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:38:25 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:38:25 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:38:25 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:38:25 Stream Closed for vb 175 StreamEnd simulated
XDCRFactory 2017-04-25T13:38:25.549-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:25.549-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:38:25.550-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:38:25.686-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:25.687-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:38:25.687-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:25.687-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:38:25.687-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:25.687-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:38:25.687-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:38:25.688-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:38:25.688-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
GenericSupervisor 2017-04-25T13:38:25.690-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:38:25.690-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
GenericSupervisor 2017-04-25T13:38:25.690-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:25.690-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:25.690-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:25.690-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:25.690-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:38:25.690-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
CheckpointManager 2017-04-25T13:38:25.690-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
CapiUtils 2017-04-25T13:38:25.789-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:25.789-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:25.789-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:38:25.790-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
StatisticsManager 2017-04-25T13:38:25.791-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:38:25.791-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
XDCRFactory 2017-04-25T13:38:25.791-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:38:25.791-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781 is constructed, start it
GenericPipeline 2017-04-25T13:38:25.792-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[230 245 277 200 272 281 285 289 305 228 243 256 283 292 176 250 252 187 246 311 198 317 231 309 236 279 295 319 323 260 288 298 219 220 173 299 203 300 212 221 239 320 326 335 325 182 188 190 232 251 210 229 274 205 186 191 235 296 313 233 291 273 282 183 225 261 267 214 280 290 328 333 302 310 234 257 312 315 177 196 237 307 322 329 179 224 254 276 337 195 223 199 275 241 227 303 304 340 181 218 331 201 208 253 270 264 321 172 215 216 258 269 180 268 294 338 332 171 238 244 174 184 185 222 226 206 248 262 301 330 336 318 306 327 197 259 271 339 178 175 193 194 249 287 308 314 247 192 213 324 189 278 209 207 211 217 263 334 202 240 242 316 266 265 284 286 297 204 255 293] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[128 159 93 6 13 4 90 102 55 23 125 136 36 73 105 108 117 134 145 27 51 35 38 14 56 138 18 19 59 86 127 140 68 76 116 144 5 34 162 147 160 37 109 1 48 131 167 11 45 50 163 166 2 111 158 41 66 115 32 57 91 9 83 157 7 8 25 155 156 31 124 10 126 70 137 21 26 98 149 44 96 150 64 71 103 52 60 79 94 164 141 146 65 75 77 87 118 143 106 112 119 132 15 113 80 84 165 154 0 67 100 130 148 81 129 42 74 120 139 39 78 110 135 85 3 170 169 17 40 46 61 72 92 97 122 29 47 142 151 24 58 82 95 99 133 12 22 28 104 152 16 107 63 88 69 89 123 33 54 168 49 53 101 62 114 20 153 30 43 121 161]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[171 238 244 184 185 222 226 174 262 301 330 336 206 248 318 306 197 259 271 327 339 175 178 287 308 314 193 194 249 192 213 247 324 189 278 209 211 217 263 334 207 240 242 316 202 266 284 286 265 204 255 293 297 245 277 230 281 285 289 200 272 243 256 283 292 305 228 176 250 252 311 187 246 198 317 231 309 279 295 319 323 236 288 298 260 219 173 220 299 203 300 212 239 320 326 335 221 188 190 232 251 325 182 210 229 274 205 186 191 235 296 313 233 291 273 183 225 261 267 282 214 290 328 333 280 310 302 315 234 257 312 177 196 237 307 322 329 179 224 254 276 195 223 337 275 199 227 241 303 304 340 331 181 218 201 208 253 270 264 321 172 215 216 258 269 180 338 268 294 332] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[3 170 169 61 72 92 97 17 40 46 122 151 29 47 142 95 99 133 24 58 82 104 152 12 22 28 16 107 63 88 69 89 123 33 54 168 49 53 101 62 114 20 153 161 30 43 121 128 159 93 6 13 4 90 102 55 23 125 136 108 117 134 145 36 73 105 27 51 35 38 14 56 138 86 127 140 18 19 59 144 68 76 116 5 34 162 147 160 37 109 167 1 48 131 11 45 50 163 166 2 111 158 41 66 115 32 57 91 9 83 157 155 156 7 8 25 31 124 10 126 70 137 149 21 26 98 44 96 150 64 71 103 94 164 52 60 79 141 146 87 118 143 65 75 77 106 112 119 132 15 113 80 84 165 154 130 148 0 67 100 81 129 139 42 74 120 39 78 110 135 85]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[checkpoint_interval:1800 optimistic_replication_threshold:256 source_nozzle_per_node:2 failure_restart_interval:10 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 stats_interval:1000]
CheckpointManager 2017-04-25T13:38:25.792-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781...
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:25.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:38:25.814-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:25.814-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:38:25.814-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:38:25.814-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:38:25.814-05:00 [INFO] Checkpointing starts in 1804.063 sec
PipelineRuntimeCtx 2017-04-25T13:38:25.814-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:38:25.814-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:38:25.814-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:38:25.815-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:38:25.816-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:38:25.816-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:38:25.816-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:38:25.816-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:38:25.817-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:38:25.843-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:38:25.843-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:38:25.843-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:38:25.848-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:38:25.848-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:38:25.886-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:38:25.888-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:38:25.892-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:25.892-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:25.892-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:38:25.892-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:38:25.892-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:25.892-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:38:25.893-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:38:25.893-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:38:25.893-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:38:25.905-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:38:25.926-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:38:25.926-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:38:25.926-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:38:25.976-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:38:25.976-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:38:26.079-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:38:26.097-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:38:26.097-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:26.097-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:38:26.098-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:38:26.098-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:38:26.098-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:38:26.098-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781 is started----------
PipelineManager 2017-04-25T13:38:26.098-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:38:26.098-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:38:26.098-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:38:26.893-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:38:27.211-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:38:27.211-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:38:27.212-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:38:27.212-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:38:27 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:38:27.329-05:00 [INFO] Send batch count=73
CapiNozzle 2017-04-25T13:38:27.350-05:00 [INFO] Send batch count=64
CapiNozzle 2017-04-25T13:38:27.372-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:27.372-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:27.382-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:27.382-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:27.400-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:38:27.411-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:38:27 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:38:27.604-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:38:27 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 25 successful
CapiNozzle 2017-04-25T13:38:27.631-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:38:27.650-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:27.687-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:38:27 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 329 successful
CapiNozzle 2017-04-25T13:38:27.749-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:27.779-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:38:27 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 240 successful
CapiNozzle 2017-04-25T13:38:27.856-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:27.874-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:38:27.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:38:27 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 247 successful
CapiNozzle 2017-04-25T13:38:27.946-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:27.946-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:38:27.946-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:38:27.946-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
ReplicationManager 2017-04-25T13:38:27.946-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
PipelineManager 2017-04-25T13:38:27.962-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:38:27.962-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:38:27.965-05:00 [INFO] err_list=[{"time":"2017-04-25T13:38:27.965169623-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]
PipelineManager 2017-04-25T13:38:27.965-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:38:27 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:38:27 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 252 successful
CapiNozzle 2017-04-25T13:38:28.054-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:28.055-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:38:28.055-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:38:28.055-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:28.055-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:28.084-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:28.084-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:28.101-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:38:28 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 64 successful
CapiNozzle 2017-04-25T13:38:28.136-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:28.136-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:28.152-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:38:28 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 291 successful
CheckpointManager 2017-04-25T13:38:28.398-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
CheckpointManager 2017-04-25T13:38:28.398-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:38:28 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 91 successful
DcpNozzle 2017-04-25T13:38:28.507-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:38:28 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:38:28 UPR_STREAMREQ for vb 98 successful
CapiNozzle 2017-04-25T13:38:28.531-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:28.543-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
DcpNozzle 2017-04-25T13:38:28.604-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:38:28.825-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:38:28.826-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781 {"Errors": "[{\"time\":\"2017-04-25T13:38:27.965169623-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 57, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 20156, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 57, "set_filtered": 0, "set_received_from_dcp": 20156, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]", "Status": "Pending"}

PipelineManager 2017-04-25T13:38:28.832-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:27.965169623-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]}
]
CapiNozzle 2017-04-25T13:38:28.833-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:28.845-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:38:29.024-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:29.043-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:38:30.182-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:30.199-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:38:30.425-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:30.425-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:38:30.425-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:38:30.425-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:30.425-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:30.939-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:30.939-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:38:30.939-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:38:30.939-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:30.939-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:30.972-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:30.972-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:31.022-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:38:31.046-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:31.046-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:31.068-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:38:31.248-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:31.264-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:38:31.513-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:31.533-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:38:31.667-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:31.680-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:38:31.977-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:31.998-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:38:32.156-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:32.156-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:38:32.156-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:38:32.156-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:32.156-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:32.207-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:32.207-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:32.223-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:38:32.477-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:32.477-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:38:32.477-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:38:32.477-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:32.477-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:32.551-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:32.552-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:32.574-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:38:32.748-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:32.768-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:38:32.989-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:33.007-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:38:33.209-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:33.241-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:38:33.442-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:33.460-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:38:33.646-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:33.646-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:38:33.646-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:38:33.646-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:33.646-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:33.744-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:33.744-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:33.766-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:38:33.823-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:33.823-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:38:33.823-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:38:33.823-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:33.823-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:33.900-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:33.900-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:33.915-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:38:34.069-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:34.100-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:38:34.189-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:34.204-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:38:34.428-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:34.440-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:38:34.591-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:34.604-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:38:34.868-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:34.868-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:38:34.868-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:38:34.868-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:34.868-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:34.929-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:34.929-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:34.949-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:38:35.036-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:35.036-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:38:35.036-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:38:35.036-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:35.036-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:35.117-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:35.117-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:35.129-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:38:35.355-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:35.366-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:38:35.702-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:35.713-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
GenericSupervisor 2017-04-25T13:38:35.815-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:38:35.815-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:38:35.815-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:38:35.899-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:35.910-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
ToplogyChangeDetector 2017-04-25T13:38:36.081-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:38:36.207-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:36.218-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:38:36.484-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:36.484-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:38:36.484-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:38:36.484-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:36.485-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:36.563-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:36.564-05:00 [ERROR] batchUpdateDocs for vb 108 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:36.574-05:00 [INFO] Retrying update docs for vb 108 for the 1th time
CapiNozzle 2017-04-25T13:38:36.697-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:36.697-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:38:36.697-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:38:36.697-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:36.697-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:36.784-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:36.784-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:36.795-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:38:36.904-05:00 [ERROR] batchUpdateDocs for vb 108 failed with err receieved unexpected status code, 500, from update docs request for vb 108
.
CapiNozzle 2017-04-25T13:38:36.915-05:00 [INFO] Retrying update docs for vb 108 for the 2th time
CapiNozzle 2017-04-25T13:38:37.106-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:37.117-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:38:37.305-05:00 [ERROR] batchUpdateDocs for vb 108 failed with err receieved unexpected status code, 500, from update docs request for vb 108
.
CapiNozzle 2017-04-25T13:38:37.316-05:00 [INFO] Retrying update docs for vb 108 for the 3th time
CapiNozzle 2017-04-25T13:38:37.704-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:37.715-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:38:37.949-05:00 [ERROR] batchUpdateDocs for vb 108 failed with err receieved unexpected status code, 500, from update docs request for vb 108
.
CapiNozzle 2017-04-25T13:38:37.949-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 108 after 3 retries
CapiNozzle 2017-04-25T13:38:37.949-05:00 [ERROR] Raise error condition batch update docs failed for vb 108 after 3 retries
GenericSupervisor 2017-04-25T13:38:37.949-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:37.949-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:38:37.965-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries 
PipelineManager 2017-04-25T13:38:37.965-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:38:37.965-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:38:37.965-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
PipelineRuntimeCtx 2017-04-25T13:38:37.965-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:38:37.965-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:37.965-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:38:37.965-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:37.965677422 -0500 CDT
GenericSupervisor 2017-04-25T13:38:37.965-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:37.965-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:38:37.965-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:38:37.965-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:38:37.965-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:38:37.965-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:38:37.965-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:38:37.965-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:38:37.966-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:38:37.966-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-687631781 has been stopped
GenericPipeline 2017-04-25T13:38:37.966-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-687631781 is stopped
PipelineManager 2017-04-25T13:38:37.966-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:38:37.966-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:27.965169623-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:38:37.966-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:38:37.966-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
GenericPipeline 2017-04-25T13:38:37.966-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
GenericPipeline 2017-04-25T13:38:37.966-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
GenericPipeline 2017-04-25T13:38:37.966-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-687631781
AsyncListener 2017-04-25T13:38:37.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:37.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:37.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:37.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:37.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:37.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:38:37.967-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:37.967-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:38:37.967-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:38:37.967-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:37.967-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:38:37.967-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:38:37.967-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:37.967-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:38:37.967-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:38:37.967-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:37.968-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:38:37.968-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:37.968-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:37.968-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:38:37.968-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:37.967808102 -0500 CDT
DcpNozzle 2017-04-25T13:38:37.968-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:38:37.968-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:37.96809306 -0500 CDT
CapiNozzle 2017-04-25T13:38:37.968-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:38:37.969-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:38:37.969-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:38:37.969-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:37.969-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:38:37.970-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:37.970-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
2017/04/25 13:38:37 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:38:37.975-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:38:37.975-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:37.97567826 -0500 CDT
DcpNozzle 2017-04-25T13:38:37.975-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:38:37.975-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:38:37.975-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:37.975788675 -0500 CDT
GenericPipeline 2017-04-25T13:38:37.975-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:37.975-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:38:37.975-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:38:37.975-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:37.975-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:38:37 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:38:37 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:38:37 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:38:37 Stream Closed for vb 174 StreamEnd simulated
CapiNozzle 2017-04-25T13:38:38.029-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:38:38.029-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:38:38.029-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:38.029-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:38:38.097-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:38.097-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:38:38.097-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:38:38.232-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:38.233-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:38:38.233-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:38.233-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:38:38.233-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:38.233-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:38:38.233-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:38:38.234-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:38:38.234-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
GenericSupervisor 2017-04-25T13:38:38.236-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:38:38.236-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
GenericSupervisor 2017-04-25T13:38:38.236-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:38.236-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:38.236-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:38.236-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:38.236-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:38:38.236-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
CheckpointManager 2017-04-25T13:38:38.236-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
CapiUtils 2017-04-25T13:38:38.261-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:38.261-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:38.261-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:38:38.262-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
StatisticsManager 2017-04-25T13:38:38.262-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:38:38.263-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
XDCRFactory 2017-04-25T13:38:38.263-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:38:38.263-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982 is constructed, start it
GenericPipeline 2017-04-25T13:38:38.264-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[12 70 57 137 143 53 105 32 51 90 104 25 87 115 162 2 98 41 77 95 169 27 69 128 149 4 16 46 146 159 71 81 88 141 89 54 85 107 135 15 39 76 121 0 150 43 82 23 124 78 99 106 9 122 34 10 11 38 168 58 93 109 5 158 22 102 142 161 66 157 14 40 48 50 86 156 96 120 29 79 108 19 64 74 59 163 44 144 13 35 45 167 152 3 7 47 118 80 97 6 52 170 123 154 62 56 94 147 136 31 60 91 126 49 73 26 36 160 67 110 139 153 117 166 24 37 55 72 65 83 132 1 20 30 61 63 101 145 18 151 33 42 103 116 133 165 140 155 164 100 130 134 138 127 84 112 125 21 129 17 92 119 68 75 114 148 8 28 113 131 111] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[234 278 191 229 242 270 172 183 301 217 226 302 218 284 182 255 206 340 194 227 203 272 304 308 313 174 175 214 222 282 246 186 306 176 244 333 189 239 305 238 249 266 322 312 201 287 338 225 254 268 180 251 193 281 252 257 309 187 221 286 289 219 292 199 230 303 216 294 202 215 228 315 314 260 274 317 330 332 179 293 331 177 231 327 258 267 277 283 185 188 208 291 236 263 210 212 285 213 259 220 171 192 195 247 235 243 288 233 339 181 190 329 209 253 261 296 279 320 207 265 276 241 298 319 324 196 204 316 326 245 280 224 321 328 335 271 310 336 264 248 290 295 205 256 318 198 311 197 173 200 275 297 307 269 334 323 250 262 273 223 237 325 178 184 240 300 337 211 232 299]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[4 16 46 71 81 88 141 146 159 89 54 85 107 135 15 39 76 121 0 150 43 82 23 124 78 99 106 9 122 34 10 11 38 168 58 93 109 5 158 22 102 142 161 14 40 48 50 66 157 86 156 96 120 29 79 108 19 64 74 59 163 44 144 13 35 45 167 3 7 47 118 152 80 97 6 52 170 123 154 62 56 94 147 31 60 91 126 136 49 73 26 36 67 110 139 153 160 24 37 55 72 117 166 132 1 20 30 61 65 83 63 101 145 18 33 42 103 116 151 133 165 164 100 130 134 138 140 155 127 84 112 125 21 129 17 92 119 68 75 114 148 8 28 113 131 111 12 70 57 137 143 53 32 51 90 104 105 25 87 115 162 2 98 41 77 95 169 27 69 128 149] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[186 246 306 176 244 333 189 239 305 266 322 238 249 312 201 287 338 225 180 251 254 268 193 281 309 187 221 252 257 219 286 289 199 230 292 216 303 294 202 215 228 315 260 274 314 332 179 293 317 330 177 331 231 327 277 283 185 188 208 258 267 291 210 212 236 263 213 259 285 220 192 195 171 247 288 233 235 243 181 190 339 209 253 261 296 329 207 265 276 279 320 319 324 196 241 298 316 326 204 245 280 224 321 328 335 271 310 336 264 248 290 295 205 256 318 311 198 197 173 200 275 297 307 269 334 323 273 223 250 262 237 325 240 300 178 184 337 211 232 299 234 278 242 270 172 183 191 229 301 217 226 302 218 284 255 182 206 194 227 340 304 308 203 272 313 214 222 174 175 282]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info replication_type:capi filter_expression: active:true checkpoint_interval:1800 failure_restart_interval:10 target_nozzle_per_node:2 worker_batch_size:500 doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:38:38.264-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982...
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:38.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:38:38.270-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:38.270-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:38:38.270-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:38:38.270-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:38:38.270-05:00 [INFO] Checkpointing starts in 1801.317 sec
PipelineRuntimeCtx 2017-04-25T13:38:38.270-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:38:38.270-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:38:38.271-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:38:38.271-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:38:38.275-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:38:38.275-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:38:38.275-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:38:38.275-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:38:38.278-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:38:38.298-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiUtils 2017-04-25T13:38:38.299-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:38.299-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:38.299-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:38:38.300-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:38:38.300-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:38:38.330-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:38:38.330-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:38:38.333-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:38:38.333-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:38:38.336-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:38:38.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:38:38.365-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:38.365-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:38:38.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:38:38.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:38:38.365-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:38:38.381-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:38:38.381-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:38:38.381-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:38:38.395-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:38:38.395-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:38:38.461-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:38:38.594-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:38:38.594-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:38.595-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:38:38.605-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:38:38.605-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:38:38.605-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:38:38.605-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982 is started----------
PipelineManager 2017-04-25T13:38:38.605-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:38:38.605-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:38:38.605-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:38:38.748-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:38:38.749-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:38:38.749-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:38:38.749-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:38:38 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:38:38.825-05:00 [INFO] Send batch count=195
2017/04/25 13:38:38 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:38:38.875-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:38.877-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:38.878-05:00 [INFO] Send batch count=42
2017/04/25 13:38:38 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:38:38.894-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:38:38.899-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:38.899-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:38:38 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 306 successful
CapiNozzle 2017-04-25T13:38:38.939-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:38:38 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:38:38 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 19 successful
CapiNozzle 2017-04-25T13:38:39.117-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
2017/04/25 13:38:39 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 217 successful
CapiNozzle 2017-04-25T13:38:39.143-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:38:39 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 33 successful
CapiNozzle 2017-04-25T13:38:39.296-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:38:39.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:38:39 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 336 successful
CapiNozzle 2017-04-25T13:38:39.350-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:39.369-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:38:39.395-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
2017/04/25 13:38:39 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 49 successful
CapiNozzle 2017-04-25T13:38:39.530-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:39.530-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:38:39.530-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:38:39.530-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
ReplicationManager 2017-04-25T13:38:39.530-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]
PipelineManager 2017-04-25T13:38:39.537-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:38:39.537-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:38:39.539-05:00 [INFO] err_list=[{"time":"2017-04-25T13:38:39.539959328-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]
PipelineManager 2017-04-25T13:38:39.540-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:38:39 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 66 successful
CapiNozzle 2017-04-25T13:38:39.709-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:39.709-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:39.726-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:38:39 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 176 successful
CapiNozzle 2017-04-25T13:38:39.748-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:39.768-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:38:39 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 96 successful
CheckpointManager 2017-04-25T13:38:39.978-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
CheckpointManager 2017-04-25T13:38:39.978-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:38:39 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:38:39 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 295 successful
CapiNozzle 2017-04-25T13:38:40.056-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:40.082-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:38:40 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:38:40 UPR_STREAMREQ for vb 98 successful
DcpNozzle 2017-04-25T13:38:40.109-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:38:40.178-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:38:40.947-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:40.948-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:40.948-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:38:40.948-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:38:40.948-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:40.948-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:38:40.962-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:38:40.962-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982 {"Errors": "[{\"time\":\"2017-04-25T13:38:39.539959328-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 89, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 17402, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 89, "set_filtered": 0, "set_received_from_dcp": 17402, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:38:41.000-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:38:41.080-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:41.080-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:41.102-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:38:41.478-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:38:41.478-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:38:41.478-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:38:41.478-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:41.478-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:41.568-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:41.568-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:41.590-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:38:41.775-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:41.788-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:38:42.007-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:42.024-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:38:42.236-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:42.249-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:38:42.378-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:42.401-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:38:42.588-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:38:42.588-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:38:42.588-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:38:42.588-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:42.588-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:42.672-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:42.672-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:42.686-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:38:42.825-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:42.825-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:38:42.825-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:38:42.825-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:42.825-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:42.885-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:42.896-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:42.915-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:38:43.036-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:43.055-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:38:43.285-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:43.302-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:38:43.427-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:43.448-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:38:43.660-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:43.678-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
PipelineManager 2017-04-25T13:38:43.792-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:39.539959328-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries]}
]
CapiNozzle 2017-04-25T13:38:43.841-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:38:43.841-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:38:43.841-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:38:43.841-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:43.841-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:43.916-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:43.916-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:43.939-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:38:44.056-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:44.056-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:38:44.056-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:38:44.056-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:44.056-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:44.146-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:44.147-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:44.162-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:38:44.357-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:44.396-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:38:44.590-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:44.607-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:38:44.777-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:44.800-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:38:45.032-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:45.052-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:38:45.219-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:45.219-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:38:45.219-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:38:45.219-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:45.219-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:45.309-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:45.309-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:45.346-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:38:45.461-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:38:45.462-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:38:45.462-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:38:45.462-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:45.462-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:45.557-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:45.557-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:45.581-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:38:45.820-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:45.841-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:38:46.006-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:46.019-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:38:46.246-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:46.265-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:38:46.485-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:46.515-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:38:46.747-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:46.747-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:38:46.747-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:38:46.747-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:46.747-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:46.874-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:46.874-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:46.889-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:38:46.907-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:46.907-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:38:46.907-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:38:46.908-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:46.908-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:46.960-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:46.961-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:46.980-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:38:47.244-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:47.258-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:38:47.422-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:47.433-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:38:47.565-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:47.577-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:38:47.759-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:47.770-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:38:47.980-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:38:47.980-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:38:47.980-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:38:47.980-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:47.980-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:48.044-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:48.044-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:48.055-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
GenericSupervisor 2017-04-25T13:38:48.271-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:38:48.271-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:38:48.271-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:38:48.380-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:48.380-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:38:48.380-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:38:48.380-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:48.380-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:38:48.490-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:38:48.522-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:48.522-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:48.533-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:38:48.870-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:38:48.881-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:38:49.494-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:38:49.505-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
PipelineManager 2017-04-25T13:38:49.551-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries 
PipelineManager 2017-04-25T13:38:49.552-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:38:49.552-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:38:49.552-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
PipelineRuntimeCtx 2017-04-25T13:38:49.552-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:38:49.552-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:49.552-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:38:49.552-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:49.552210243 -0500 CDT
GenericSupervisor 2017-04-25T13:38:49.552-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:49.552-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:38:49.552-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:38:49.552-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:38:49.552-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:38:49.552-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:38:49.552-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:38:49.552-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:38:49.552-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:38:49.552-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-233899982 has been stopped
GenericPipeline 2017-04-25T13:38:49.552-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-233899982 is stopped
PipelineManager 2017-04-25T13:38:49.552-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:38:49.552-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:39.539959328-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 301 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:38:49.553-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:38:49.553-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
GenericPipeline 2017-04-25T13:38:49.553-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
GenericPipeline 2017-04-25T13:38:49.553-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
GenericPipeline 2017-04-25T13:38:49.553-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-233899982
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:38:49.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:38:49.553-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:49.553-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:38:49.553-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:49.554-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:38:49.554-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:49.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:38:49.554-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:38:49.554-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:38:49.554-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:38:49.554-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:38:49.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:38:49.554-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:49.554145695 -0500 CDT
DcpNozzle 2017-04-25T13:38:49.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:38:49.554-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:49.554470422 -0500 CDT
CapiNozzle 2017-04-25T13:38:49.555-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:38:49.555-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:38:49.555-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:38:49.555-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:38:49.555-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:38:49.556-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:49.556-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:38:49.558-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:38:49.558-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:38:49.558-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:49.558900685 -0500 CDT
DcpNozzle 2017-04-25T13:38:49.558-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:38:49.558915977 -0500 CDT
DcpNozzle 2017-04-25T13:38:49.559-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:38:49.559-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:38:49.559-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:49.559-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:38:49.559-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:49.559-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:38:49 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:38:49 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:38:49 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:38:49.570-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:38:49.570-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:38:49.570-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:38:49.570-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:38:49.901-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:49.901-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:38:49.901-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:38:49.944-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:38:49.945-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:38:49.945-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:49.945-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:38:49.945-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:38:49.945-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:38:49.945-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:38:49.945-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:38:49.946-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
GenericSupervisor 2017-04-25T13:38:49.948-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:38:49.948-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
GenericSupervisor 2017-04-25T13:38:49.948-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:49.948-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:49.948-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:49.948-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:38:49.948-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:38:49.948-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
CheckpointManager 2017-04-25T13:38:49.948-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
CapiUtils 2017-04-25T13:38:49.978-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:49.978-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:49.978-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:38:49.979-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
StatisticsManager 2017-04-25T13:38:49.979-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:38:49.979-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
XDCRFactory 2017-04-25T13:38:49.979-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:38:49.979-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715 is constructed, start it
GenericPipeline 2017-04-25T13:38:49.981-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[191 194 205 302 207 235 261 249 326 286 314 328 325 280 210 277 288 312 329 339 251 208 245 250 259 296 172 196 266 298 239 217 218 264 219 174 211 227 282 291 306 265 281 272 274 244 313 309 186 287 308 200 201 333 222 226 310 336 228 284 323 178 223 279 303 190 234 206 209 278 254 315 224 214 221 275 173 230 238 268 304 295 177 289 321 322 324 171 220 233 337 305 192 195 240 327 189 232 267 269 182 252 273 283 242 338 340 180 318 183 188 256 316 184 215 225 334 237 258 317 300 319 243 212 202 246 197 204 236 176 198 216 276 320 307 241 293 330 285 247 187 290 301 193 181 229 257 292 332 335 262 175 199 311 248 297 179 185 231 253 263 331 270 260 213 299 255 203 271 294] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[61 18 130 134 144 5 128 112 29 70 87 102 113 28 147 153 8 68 95 122 50 30 54 106 116 118 20 47 78 108 110 133 10 85 64 94 101 109 14 83 145 41 156 80 91 34 82 45 158 146 103 141 163 62 111 136 165 81 37 74 143 7 38 49 63 96 154 25 132 32 46 42 149 53 22 40 125 9 159 170 55 137 152 155 164 48 157 168 75 60 104 119 124 139 36 17 123 1 98 56 120 11 58 89 115 162 51 27 92 4 88 93 97 135 142 35 86 99 19 71 107 131 151 2 31 66 127 148 6 52 72 77 12 117 43 140 26 16 13 79 126 0 161 73 44 65 129 160 15 24 150 169 3 121 57 167 90 114 166 67 69 23 59 33 76 21 84 100 105 138 39]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[43 117 140 16 26 0 13 79 126 73 161 15 44 65 129 160 3 24 150 169 57 121 90 167 67 114 166 23 69 59 21 33 76 39 84 100 105 138 61 5 18 130 134 144 112 128 29 28 70 87 102 113 147 8 153 68 50 95 122 20 30 54 106 116 118 10 47 78 108 110 133 85 14 64 94 101 109 41 83 145 80 156 34 91 45 82 146 158 62 103 141 163 81 111 136 165 7 37 74 143 25 38 49 63 96 154 32 132 42 46 53 149 9 22 40 125 159 170 48 55 137 152 155 164 75 157 168 36 60 104 119 124 139 17 1 123 56 98 11 120 58 51 89 115 162 4 27 92 88 35 93 97 135 142 19 86 99 2 71 107 131 151 6 31 66 127 148 12 52 72 77] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[241 293 330 285 187 247 290 301 193 229 257 181 262 292 332 335 311 175 199 248 297 253 263 179 185 231 270 331 213 260 299 255 203 271 294 191 194 205 302 235 261 207 249 326 286 314 328 325 280 329 339 210 277 288 312 251 208 296 245 250 259 172 196 239 266 298 217 218 219 264 306 174 211 227 282 291 281 265 244 272 274 309 313 186 287 308 200 201 333 222 226 310 336 323 228 284 279 303 178 223 190 234 206 209 278 254 315 224 214 221 275 238 268 173 230 295 304 177 289 321 322 324 171 220 233 305 337 192 195 240 327 189 232 267 269 182 252 273 283 242 338 340 180 318 316 183 188 256 184 215 225 334 258 317 237 300 319 212 243 246 202 204 236 197 216 276 176 198 307 320]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi active:true log_level:Info stats_interval:1000 filter_expression:]
CheckpointManager 2017-04-25T13:38:49.981-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715...
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:38:49.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:38:49.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
StatisticsManager 2017-04-25T13:38:49.989-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:38:49.994-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:38:49.994-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:38:49.994-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:38:49.994-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:38:49.994-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:38:49.994-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:38:49.994-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:38:49.994-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:38:49.994-05:00 [INFO] Checkpointing starts in 1804.548 sec
PipelineRuntimeCtx 2017-04-25T13:38:49.994-05:00 [INFO] Service CheckpointManager has been started
GenericSupervisor 2017-04-25T13:38:49.994-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:38:49.994-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:38:49.995-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:38:50.033-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:38:50.033-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:38:50.033-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:38:50.051-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:38:50.051-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:38:50.055-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:38:50.055-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:38:50.055-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:38:50.055-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:38:50.064-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:38:50.065-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:38:50.072-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:38:50.072-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:50.072-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:38:50.072-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:38:50.073-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:38:50.073-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:38:50.131-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:38:50.131-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:38:50.131-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:38:50.142-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:38:50.142-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:38:50.199-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:38:50.268-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:38:50.268-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:38:50.268-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:38:50.268-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:38:50.268-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:38:50.268-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:38:50.268-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715 is started----------
PipelineManager 2017-04-25T13:38:50.269-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:38:50.269-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:38:50.269-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:38:50.433-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:38:50.433-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:38:50.433-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:38:50.433-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:38:50 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:38:50.518-05:00 [INFO] Send batch count=55
CapiNozzle 2017-04-25T13:38:50.567-05:00 [INFO] Send batch count=238
2017/04/25 13:38:50 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 6 successful
CapiNozzle 2017-04-25T13:38:50.654-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:50.654-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:50.673-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
2017/04/25 13:38:50 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 111 successful
CapiNozzle 2017-04-25T13:38:50.712-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:50.743-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:38:50 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 323 successful
CapiNozzle 2017-04-25T13:38:50.885-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:38:50 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 23 successful
CapiNozzle 2017-04-25T13:38:50.916-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:38:50 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:38:50 UPR_STREAMREQ for vb 236 successful
StatisticsManager 2017-04-25T13:38:51.035-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:38:51 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 37 successful
CapiNozzle 2017-04-25T13:38:51.169-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:38:51.185-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
2017/04/25 13:38:51 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:38:51.277-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:38:51 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 263 successful
CapiNozzle 2017-04-25T13:38:51.304-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:38:51 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 95 successful
CapiNozzle 2017-04-25T13:38:51.493-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:38:51 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 191 successful
CapiNozzle 2017-04-25T13:38:51.499-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:51.499-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:38:51.499-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:38:51.499-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:38:51.499-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
CapiNozzle 2017-04-25T13:38:51.524-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
PipelineManager 2017-04-25T13:38:51.539-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:38:51.539-05:00 [INFO] Send batch count=500
CheckpointManager 2017-04-25T13:38:51.541-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
CheckpointManager 2017-04-25T13:38:51.541-05:00 [INFO] Done with SetVBTimestamps
PipelineManager 2017-04-25T13:38:51.541-05:00 [INFO] err_list=[{"time":"2017-04-25T13:38:51.541753678-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:38:51.541-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
DcpNozzle 2017-04-25T13:38:51.594-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:38:51.608-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:51.608-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:38:51 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:38:51 UPR_STREAMREQ for vb 297 successful
CapiNozzle 2017-04-25T13:38:51.621-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
DcpNozzle 2017-04-25T13:38:51.681-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:38:51.903-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:38:51.903-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:38:51.903-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:38:51.903-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:51.903-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:51.996-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:51.996-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:38:52.001-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:38:52.001-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715 {"Errors": "[{\"time\":\"2017-04-25T13:38:51.541753678-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 135, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 21366, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 135, "set_filtered": 0, "set_received_from_dcp": 21366, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:38:52.014-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:38:52.212-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:52.228-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:38:52.627-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:52.640-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:38:52.805-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:52.816-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:38:52.968-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:52.986-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:38:53.190-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:38:53.190-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:38:53.190-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:38:53.190-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:53.190-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:53.254-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:53.254-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:53.265-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:38:53.337-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:38:53.337-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:38:53.337-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:38:53.337-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:53.338-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:53.432-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:53.432-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:53.443-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:38:53.674-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:53.688-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:38:53.830-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:53.843-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:38:55.003-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:55.029-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:38:55.238-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:55.264-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:38:55.452-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:38:55.452-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:38:55.452-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:38:55.452-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:55.452-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:55.536-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:55.536-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:55.566-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:38:55.592-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:38:55.592-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:38:55.592-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:38:55.592-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:55.592-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:55.690-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:55.690-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:55.708-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:38:56.115-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:56.151-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:38:56.420-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:38:56.439-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:38:56.634-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:56.648-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:38:57.021-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:38:57.044-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:38:57.378-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:38:57.400-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:38:57.400-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:38:57.400-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:57.400-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:57.490-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:57.491-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:57.514-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:38:57.583-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:38:57.584-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:38:57.584-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:38:57.584-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:38:57.584-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:38:57.664-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:38:57.665-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:38:57.681-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:38:58.091-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:58.110-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:38:58.586-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:58.608-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
PipelineManager 2017-04-25T13:38:58.764-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:51.541753678-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
ReplicationManager 2017-04-25T13:38:58.926-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:38:58.934-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:38:58.960-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:38:59.536-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:38:59.547-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
GenericSupervisor 2017-04-25T13:38:59.995-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:38:59.995-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:38:59.995-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:39:00.249-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:39:00.870-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:00.870-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:39:00.870-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:39:00.870-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:00.870-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:00.953-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:00.953-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:00.964-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:39:01.279-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:39:01.279-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:39:01.280-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:39:01.280-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:01.280-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:01.362-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:01.362-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:01.375-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
PipelineManager 2017-04-25T13:39:01.542-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:39:01.542-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:39:01.542-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:01.542-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
PipelineRuntimeCtx 2017-04-25T13:39:01.542-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:39:01.542-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:01.542-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:39:01.542-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:01.542300711 -0500 CDT
GenericSupervisor 2017-04-25T13:39:01.542-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:01.542-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:39:01.542-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:01.542-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:39:01.542-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:01.542-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:39:01.542-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:39:01.542-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:39:01.542-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:39:01.542-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-945489715 has been stopped
GenericPipeline 2017-04-25T13:39:01.542-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-945489715 is stopped
PipelineManager 2017-04-25T13:39:01.542-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:39:01.542-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:39:01.542-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
GenericPipeline 2017-04-25T13:39:01.542-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
PipelineManager 2017-04-25T13:39:01.542-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:38:51.541753678-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:39:01.543-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
GenericPipeline 2017-04-25T13:39:01.543-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-945489715
PipelineManager 2017-04-25T13:39:01.543-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:01.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:39:01.543-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:39:01.543-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:39:01.543-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:39:01.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:01.543-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:01.543-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:01.543-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:39:01.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:39:01.543-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:01.543501447 -0500 CDT
DcpNozzle 2017-04-25T13:39:01.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.543-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:01.543774688 -0500 CDT
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:39:01.544-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:01.544-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:01.544-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:39:01.544-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:01.544-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:39:01.546-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:01.546-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:01.546-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:01.546332181 -0500 CDT
DcpNozzle 2017-04-25T13:39:01.546-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:39:01.546-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:01.546342948 -0500 CDT
DcpNozzle 2017-04-25T13:39:01.546-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:39:01.546-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:01.546-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:39:01.546-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:01.546-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:39:01 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 4 StreamEnd simulated
2017/04/25 13:39:01 Stream Closed for vb 5 StreamEnd simulated
XDCRFactory 2017-04-25T13:39:01.667-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:01.668-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:39:01.668-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:39:01.869-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:01.869-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:39:01.869-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:01.869-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:39:01.869-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:01.869-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:39:01.869-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:39:01.870-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:39:01.870-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
GenericSupervisor 2017-04-25T13:39:01.872-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:39:01.872-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
GenericSupervisor 2017-04-25T13:39:01.872-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:01.872-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:01.872-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:01.872-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:01.872-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:39:01.872-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
CheckpointManager 2017-04-25T13:39:01.872-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
CapiUtils 2017-04-25T13:39:01.970-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:01.970-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:01.970-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:39:01.971-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
StatisticsManager 2017-04-25T13:39:01.972-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:39:01.972-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
XDCRFactory 2017-04-25T13:39:01.972-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:39:01.972-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314 is constructed, start it
GenericPipeline 2017-04-25T13:39:01.974-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[145 147 7 39 75 76 134 73 63 109 30 33 61 40 100 118 3 121 133 148 105 131 167 14 46 71 72 96 8 60 142 20 81 143 25 38 78 159 17 149 154 165 44 42 106 123 70 101 125 28 50 110 166 115 117 27 35 95 15 54 113 16 97 4 12 23 89 45 158 66 99 160 5 126 114 150 153 22 48 52 168 10 92 94 2 93 157 11 58 102 137 136 162 79 43 53 112 128 161 56 144 68 90 170 88 169 86 108 155 85 34 47 80 84 122 135 57 103 24 129 29 98 140 67 9 26 91 116 124 62 65 138 120 13 83 156 104 151 36 64 74 77 82 59 69 107 6 21 32 41 51 119 141 130 1 55 139 152 127 132 146 87 111 164 49 18 37 163 0 19 31] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[213 220 259 323 202 257 181 204 283 280 290 205 224 241 266 222 308 262 270 322 180 203 339 261 235 211 333 193 218 307 175 179 328 251 263 335 194 173 255 258 312 284 186 248 225 229 296 198 297 317 289 309 325 329 340 183 313 172 240 253 232 274 227 300 187 271 295 337 275 326 230 273 332 281 192 242 282 237 200 212 244 206 331 171 207 239 214 174 221 228 256 277 315 210 219 264 191 215 188 197 320 246 265 276 196 234 302 319 182 311 330 208 286 293 334 250 285 301 223 201 245 292 338 184 233 310 247 254 298 306 178 209 252 260 269 318 287 294 321 185 216 288 291 336 314 303 305 217 327 189 324 272 278 236 249 267 279 238 304 316 177 199 268 176 243 226 231 299 190 195]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[306 247 254 298 178 209 252 260 269 318 216 287 294 321 185 288 291 336 314 303 305 217 327 189 324 267 272 278 236 249 279 199 238 304 316 177 243 268 176 226 231 195 299 190 213 220 257 259 323 202 283 181 204 266 280 290 205 224 241 222 308 203 262 270 322 180 339 261 235 211 333 179 193 218 307 175 328 263 251 335 173 194 255 258 312 248 284 186 198 225 229 296 297 317 325 289 309 313 329 340 183 240 172 253 227 232 274 295 300 187 271 326 337 275 332 230 273 242 281 192 237 282 200 244 212 206 239 331 171 207 214 174 221 228 256 277 264 315 210 219 215 191 320 188 197 234 246 265 276 196 302 319 182 311 330 208 286 293 334 250 285 301 223 292 201 245 338 184 233 310] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[67 124 9 26 91 116 62 65 138 120 13 83 156 82 104 151 36 64 74 77 51 59 69 107 6 21 32 41 119 141 130 1 55 139 152 127 132 146 87 111 164 49 18 37 163 0 19 31 145 147 7 39 75 76 134 73 63 109 30 33 61 40 100 118 3 121 133 148 96 105 131 167 14 46 71 72 8 60 142 20 81 143 25 38 78 159 17 149 154 165 44 42 106 123 70 101 125 28 50 110 166 115 117 27 35 95 15 54 113 16 97 4 12 23 89 45 158 66 99 160 5 126 114 150 153 22 48 52 168 10 92 94 2 93 157 11 58 102 137 136 162 79 161 43 53 112 128 56 144 68 90 170 88 169 86 108 155 85 34 47 80 84 122 135 57 103 24 129 29 98 140]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: source_nozzle_per_node:2 target_nozzle_per_node:2 active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:39:01.974-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314...
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:39:01.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:39:01.996-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:01.996-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:39:01.996-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:39:01.996-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:39:01.996-05:00 [INFO] Checkpointing starts in 1802.474 sec
PipelineRuntimeCtx 2017-04-25T13:39:01.996-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:39:01.996-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:39:01.997-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:39:01.997-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:39:02.001-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:39:02.001-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:39:02.001-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:39:02.001-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:39:02.005-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:39:02.034-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:39:02.034-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:39:02.034-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:39:02.066-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:39:02.069-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:02.069-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:02.069-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:39:02.073-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:39:02.073-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:39:02.084-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:39:02.092-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:39:02.149-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:39:02.155-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:39:02.155-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:02.155-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:39:02.155-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:39:02.155-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:39:02.155-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:39:02.171-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:39:02.171-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:39:02.171-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:39:02.187-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:39:02.187-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:39:02.342-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:39:02.342-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:39:02.342-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:39:02.342-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:39:02.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:39:02.365-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:02.365-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:39:02.391-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:39:02.391-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:39:02.391-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:39:02.391-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314 is started----------
PipelineManager 2017-04-25T13:39:02.391-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:39:02.391-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:39:02.391-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:39:02 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:39:02.414-05:00 [INFO] Send batch count=2
2017/04/25 13:39:02 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 324 successful
CapiNozzle 2017-04-25T13:39:02.658-05:00 [INFO] Send batch count=42
2017/04/25 13:39:02 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 38 successful
CapiNozzle 2017-04-25T13:39:02.690-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:02.690-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:39:02 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:39:02.732-05:00 [INFO] Retrying update docs for vb 216 for the 1th time
CapiNozzle 2017-04-25T13:39:02.740-05:00 [ERROR] Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f100/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:39:02.740-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err Received malformed response from tcp connection vb=0. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f100/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:39:02.767-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:39:02 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:39:02 UPR_STREAMREQ for vb 255 successful
StatisticsManager 2017-04-25T13:39:03.068-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:39:03 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 261 successful
CapiNozzle 2017-04-25T13:39:03.163-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:39:03 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 68 successful
CapiNozzle 2017-04-25T13:39:03.183-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:39:03 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 180 successful
CapiNozzle 2017-04-25T13:39:03.344-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err receieved unexpected status code, 500, from update docs request for vb 216
.
CapiNozzle 2017-04-25T13:39:03.360-05:00 [INFO] Retrying update docs for vb 216 for the 2th time
2017/04/25 13:39:03 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 292 successful
CapiNozzle 2017-04-25T13:39:03.630-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CheckpointManager 2017-04-25T13:39:03.637-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
CheckpointManager 2017-04-25T13:39:03.637-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:39:03.647-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
DcpNozzle 2017-04-25T13:39:03.660-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:39:03 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:39:03 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:39:03.757-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err receieved unexpected status code, 500, from update docs request for vb 216
.
CapiNozzle 2017-04-25T13:39:03.773-05:00 [INFO] Retrying update docs for vb 216 for the 3th time
DcpNozzle 2017-04-25T13:39:03.796-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:39:03.954-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:03.954-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:03.954-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:03.954-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:39:03.954-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:39:03.974-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:39:03.974-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:39:03.977-05:00 [INFO] err_list=[{"time":"2017-04-25T13:39:03.977388495-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:39:03.977-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:39:04.017-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:39:04.017-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314 {"Errors": "[{\"time\":\"2017-04-25T13:39:03.977388495-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 1, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 22960, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 1, "set_filtered": 0, "set_received_from_dcp": 22960, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:39:04.034-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:04.037-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:04.052-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:39:04.058-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err receieved unexpected status code, 500, from update docs request for vb 216
.
CapiNozzle 2017-04-25T13:39:04.058-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 216 after 3 retries
CapiNozzle 2017-04-25T13:39:04.058-05:00 [ERROR] Raise error condition batch update docs failed for vb 216 after 3 retries
GenericSupervisor 2017-04-25T13:39:04.058-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:04.058-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:04.169-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:04.169-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:04.196-05:00 [INFO] Retrying update docs for vb 211 for the 1th time
CapiNozzle 2017-04-25T13:39:05.070-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:05.089-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:39:05.233-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:39:05.253-05:00 [INFO] Retrying update docs for vb 211 for the 2th time
CapiNozzle 2017-04-25T13:39:06.306-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:06.331-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:39:06.477-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:39:06.490-05:00 [INFO] Retrying update docs for vb 211 for the 3th time
CapiNozzle 2017-04-25T13:39:07.482-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:07.482-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:07.482-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:07.482-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:07.482-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:07.566-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:07.566-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:07.617-05:00 [INFO] Retrying update docs for vb 106 for the 1th time
CapiNozzle 2017-04-25T13:39:07.692-05:00 [ERROR] batchUpdateDocs for vb 211 failed with err receieved unexpected status code, 500, from update docs request for vb 211
.
CapiNozzle 2017-04-25T13:39:07.692-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 211 after 3 retries
CapiNozzle 2017-04-25T13:39:07.692-05:00 [ERROR] Raise error condition batch update docs failed for vb 211 after 3 retries
GenericSupervisor 2017-04-25T13:39:07.692-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:07.692-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:07.719-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 314
CapiNozzle 2017-04-25T13:39:07.781-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:07.782-05:00 [ERROR] batchUpdateDocs for vb 314 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:07.798-05:00 [INFO] Retrying update docs for vb 314 for the 1th time
CapiNozzle 2017-04-25T13:39:08.416-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:39:08.428-05:00 [INFO] Retrying update docs for vb 106 for the 2th time
CapiNozzle 2017-04-25T13:39:08.772-05:00 [ERROR] batchUpdateDocs for vb 314 failed with err receieved unexpected status code, 500, from update docs request for vb 314
.
CapiNozzle 2017-04-25T13:39:08.790-05:00 [INFO] Retrying update docs for vb 314 for the 2th time
CapiNozzle 2017-04-25T13:39:09.347-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:39:09.368-05:00 [INFO] Retrying update docs for vb 106 for the 3th time
CapiNozzle 2017-04-25T13:39:09.885-05:00 [ERROR] batchUpdateDocs for vb 314 failed with err receieved unexpected status code, 500, from update docs request for vb 314
.
CapiNozzle 2017-04-25T13:39:09.903-05:00 [INFO] Retrying update docs for vb 314 for the 3th time
CapiNozzle 2017-04-25T13:39:10.328-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:39:10.328-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 106 after 3 retries
CapiNozzle 2017-04-25T13:39:10.328-05:00 [ERROR] Raise error condition batch update docs failed for vb 106 after 3 retries
GenericSupervisor 2017-04-25T13:39:10.328-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:10.328-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:10.403-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:10.403-05:00 [ERROR] batchUpdateDocs for vb 9 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:10.414-05:00 [INFO] Retrying update docs for vb 9 for the 1th time
CapiNozzle 2017-04-25T13:39:10.692-05:00 [ERROR] batchUpdateDocs for vb 314 failed with err receieved unexpected status code, 500, from update docs request for vb 314
.
CapiNozzle 2017-04-25T13:39:10.692-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 314 after 3 retries
CapiNozzle 2017-04-25T13:39:10.692-05:00 [ERROR] Raise error condition batch update docs failed for vb 314 after 3 retries
GenericSupervisor 2017-04-25T13:39:10.692-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:10.692-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:10.720-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 216
CapiNozzle 2017-04-25T13:39:10.792-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:10.792-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:10.803-05:00 [INFO] Retrying update docs for vb 216 for the 1th time
CapiNozzle 2017-04-25T13:39:10.825-05:00 [ERROR] batchUpdateDocs for vb 9 failed with err receieved unexpected status code, 500, from update docs request for vb 9
.
CapiNozzle 2017-04-25T13:39:10.839-05:00 [INFO] Retrying update docs for vb 9 for the 2th time
CapiNozzle 2017-04-25T13:39:11.380-05:00 [ERROR] batchUpdateDocs for vb 9 failed with err receieved unexpected status code, 500, from update docs request for vb 9
.
CapiNozzle 2017-04-25T13:39:11.392-05:00 [INFO] Retrying update docs for vb 9 for the 3th time
CapiNozzle 2017-04-25T13:39:11.710-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err receieved unexpected status code, 500, from update docs request for vb 216
.
CapiNozzle 2017-04-25T13:39:11.721-05:00 [INFO] Retrying update docs for vb 216 for the 2th time
GenericSupervisor 2017-04-25T13:39:11.997-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:39:11.997-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:39:11.997-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:39:12.188-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:39:12.392-05:00 [ERROR] batchUpdateDocs for vb 9 failed with err receieved unexpected status code, 500, from update docs request for vb 9
.
CapiNozzle 2017-04-25T13:39:12.392-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 9 after 3 retries
CapiNozzle 2017-04-25T13:39:12.392-05:00 [ERROR] Raise error condition batch update docs failed for vb 9 after 3 retries
GenericSupervisor 2017-04-25T13:39:12.392-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:12.392-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:12.423-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 14
CapiNozzle 2017-04-25T13:39:12.512-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:12.512-05:00 [ERROR] batchUpdateDocs for vb 14 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:12.523-05:00 [INFO] Retrying update docs for vb 14 for the 1th time
CapiNozzle 2017-04-25T13:39:12.991-05:00 [ERROR] batchUpdateDocs for vb 216 failed with err receieved unexpected status code, 500, from update docs request for vb 216
.
CapiNozzle 2017-04-25T13:39:13.002-05:00 [INFO] Retrying update docs for vb 216 for the 3th time
PipelineManager 2017-04-25T13:39:13.758-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:03.977388495-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:39:13.799-05:00 [ERROR] batchUpdateDocs for vb 14 failed with err receieved unexpected status code, 500, from update docs request for vb 14
.
CapiNozzle 2017-04-25T13:39:13.810-05:00 [INFO] Retrying update docs for vb 14 for the 2th time
PipelineManager 2017-04-25T13:39:13.977-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:39:13.977-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:39:13.977-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:13.977-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
PipelineRuntimeCtx 2017-04-25T13:39:13.977-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:39:13.977-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:13.977-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:39:13.977-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:13.977877723 -0500 CDT
GenericSupervisor 2017-04-25T13:39:13.977-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:13.978-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:39:13.978-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:13.978-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:39:13.978-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:13.978-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:39:13.978-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:39:13.978-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:39:13.978-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:39:13.978-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-869969314 has been stopped
GenericPipeline 2017-04-25T13:39:13.978-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-869969314 is stopped
PipelineManager 2017-04-25T13:39:13.978-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:39:13.978-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:03.977388495-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:39:13.978-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:13.978-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
GenericPipeline 2017-04-25T13:39:13.979-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
GenericPipeline 2017-04-25T13:39:13.979-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
GenericPipeline 2017-04-25T13:39:13.979-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-869969314
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:13.979-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:39:13.979-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:13.979-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:39:13.979-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:13.980-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:39:13.980-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:13.980-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:39:13.980-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:13.980-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:39:13.980-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:13.980-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:13.980-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:39:13.980-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:13.980161981 -0500 CDT
DcpNozzle 2017-04-25T13:39:13.980-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:39:13.981-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:13.980609588 -0500 CDT
CapiNozzle 2017-04-25T13:39:13.982-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:39:13.982-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:13.983-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:39:13.983-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:13.984-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:39:13.984-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
2017/04/25 13:39:13 Stream Closed for vb 171 StreamEnd simulated
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:13.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:13.984-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:39:13.984-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:13.984-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:39:13.986-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:13.986-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:13.986191983 -0500 CDT
DcpNozzle 2017-04-25T13:39:13.986-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:39:13.986-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:13.986-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:39:13.986-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:39:13 Stream Closed for vb 172 StreamEnd simulated
DcpNozzle 2017-04-25T13:39:13.986-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:13.986393132 -0500 CDT
DcpNozzle 2017-04-25T13:39:13.986-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:39:13.986-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:13.986-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:39:13 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:39:14.102-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:14.103-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:39:14.103-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:39:14.331-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:14.331-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:39:14.331-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:14.331-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:39:14.331-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:14.331-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:39:14.331-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:39:14.332-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:39:14.332-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
GenericSupervisor 2017-04-25T13:39:14.334-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:39:14.334-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
GenericSupervisor 2017-04-25T13:39:14.334-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:14.334-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:14.334-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:14.334-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:14.334-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:39:14.334-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
CheckpointManager 2017-04-25T13:39:14.334-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
CapiUtils 2017-04-25T13:39:14.397-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:14.397-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:14.397-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:39:14.397-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
StatisticsManager 2017-04-25T13:39:14.398-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:39:14.398-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
XDCRFactory 2017-04-25T13:39:14.398-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:39:14.398-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349 is constructed, start it
GenericPipeline 2017-04-25T13:39:14.399-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[122 147 25 48 64 30 103 45 46 55 130 132 164 165 70 74 115 89 63 79 118 167 101 133 159 33 47 11 31 149 16 58 163 90 35 52 60 56 94 158 36 49 80 2 26 117 139 142 0 24 50 114 144 99 136 23 43 73 102 12 59 66 93 69 143 125 138 157 51 78 128 10 15 107 150 37 42 100 112 116 65 106 119 154 39 81 97 113 137 170 168 41 87 13 129 1 17 134 7 18 40 109 124 156 160 84 96 153 151 155 88 126 161 32 131 148 27 108 34 71 82 57 83 127 19 111 3 8 22 86 20 77 123 14 62 67 166 85 92 104 146 28 68 75 98 105 121 140 169 38 91 110 141 29 135 44 72 4 5 21 61 95 120 152 6 54 145 9 53 76 162] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[171 254 267 287 182 291 208 228 251 319 242 203 216 178 226 237 265 300 325 332 173 195 239 248 236 316 326 198 337 177 209 222 235 292 174 317 189 207 269 315 197 219 278 297 340 196 275 314 320 321 323 199 240 282 175 224 333 281 176 179 187 277 330 200 298 220 256 186 212 293 241 255 274 172 233 247 227 210 211 286 303 331 185 243 249 310 318 328 339 276 229 245 205 230 234 192 193 246 294 266 322 335 313 250 280 214 289 307 201 223 225 258 261 279 180 217 238 244 268 327 188 252 288 309 324 183 263 272 329 273 283 284 306 308 232 264 271 296 302 285 190 206 221 218 231 184 191 260 336 257 290 299 311 213 301 194 270 295 253 334 262 215 304 305 338 202 312 181 204 259]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[249 310 318 328 339 185 243 276 229 245 234 205 230 193 192 294 246 266 322 335 313 250 280 289 307 214 201 223 225 258 261 279 217 238 180 268 327 244 288 309 324 188 252 183 263 272 329 283 284 306 308 273 232 264 271 296 302 285 190 206 221 218 231 184 191 260 336 299 311 257 290 213 301 295 194 270 253 334 262 304 305 338 215 202 312 181 204 259 254 267 171 287 291 182 208 228 251 319 242 203 216 178 226 237 265 325 332 300 173 195 239 248 326 236 316 198 337 222 235 292 177 209 174 317 315 189 207 269 197 219 278 297 340 196 275 314 320 321 323 282 199 240 175 224 333 281 179 187 277 330 176 200 298 256 220 212 293 186 255 241 274 172 233 247 227 210 211 286 303 331] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[116 154 65 106 119 113 137 170 39 81 97 168 41 87 13 129 1 17 134 109 124 156 160 7 18 40 84 96 153 151 155 88 126 161 32 131 148 27 108 34 71 82 57 83 127 19 111 86 3 8 22 20 77 123 14 62 67 166 146 85 92 104 28 68 75 98 105 121 140 169 141 38 91 110 29 135 44 72 61 95 120 152 4 5 21 6 54 145 162 9 53 76 122 147 25 48 64 30 103 130 132 164 165 45 46 55 70 74 115 89 167 63 79 118 101 133 159 33 47 11 31 149 16 58 163 90 35 52 60 56 94 158 36 49 80 139 142 2 26 117 0 144 24 50 114 99 136 102 23 43 73 12 59 66 93 69 143 125 138 157 51 78 128 10 15 107 150 112 37 42 100]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: worker_batch_size:500 log_level:Info active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:39:14.400-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349...
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:14.418-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:39:14.418-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:14.418-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:39:14.418-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:39:14.418-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:39:14.418-05:00 [INFO] Checkpointing starts in 1802.889 sec
PipelineRuntimeCtx 2017-04-25T13:39:14.418-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:39:14.418-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:39:14.418-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:39:14.419-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:39:14.419-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:39:14.419-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:39:14.419-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:39:14.419-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:39:14.423-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:39:14.451-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:39:14.451-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:39:14.451-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:39:14.469-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:39:14.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:39:14.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:39:14.490-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:39:14.495-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:14.495-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:14.495-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:39:14.506-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:39:14.507-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:39:14.508-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:14.508-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:39:14.508-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:39:14.509-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:39:14.509-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:39:14.548-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:39:14.548-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:39:14.548-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:39:14.585-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:39:14.585-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:39:14.674-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:39:14.674-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:14.674-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:39:14.674-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:39:14.674-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:39:14.674-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:39:14.674-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349 is started----------
PipelineManager 2017-04-25T13:39:14.674-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:39:14.674-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:39:14.674-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:39:14.687-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:39:14.997-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:39:14.997-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:39:14.997-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:39:14.997-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:39:15 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:39:15.052-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:39:15.061-05:00 [INFO] Send batch count=127
CapiNozzle 2017-04-25T13:39:15.087-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:15.087-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:39:15 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:39:15.118-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:39:15.152-05:00 [INFO] Send batch count=36
2017/04/25 13:39:15 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 32 successful
CapiNozzle 2017-04-25T13:39:15.446-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:15.461-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
StatisticsManager 2017-04-25T13:39:15.466-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:39:15 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 245 successful
CapiNozzle 2017-04-25T13:39:15.717-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:39:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 56 successful
CapiNozzle 2017-04-25T13:39:15.745-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:39:15 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 81 successful
CapiNozzle 2017-04-25T13:39:15.976-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:15.976-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:15.976-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:15.976-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:39:15.976-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:39:15 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:39:15 UPR_STREAMREQ for vb 183 successful
PipelineManager 2017-04-25T13:39:15.991-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:39:15.991-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:39:15.995-05:00 [INFO] err_list=[{"time":"2017-04-25T13:39:15.995304592-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:39:15.995-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:39:16 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:39:16.067-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:16.067-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CheckpointManager 2017-04-25T13:39:16.079-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
CheckpointManager 2017-04-25T13:39:16.079-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:39:16.082-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:39:16 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:39:16 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:39:16.218-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:39:16.284-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:39:16.416-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
StatisticsManager 2017-04-25T13:39:16.421-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:39:16.421-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349 {"Errors": "[{\"time\":\"2017-04-25T13:39:15.995304592-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 71, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 13411, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 71, "set_filtered": 0, "set_received_from_dcp": 13411, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:39:16.442-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:39:16.777-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:39:16.793-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:39:17.388-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:39:17.388-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:39:17.388-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:39:17.388-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:17.388-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:17.502-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:17.502-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:17.530-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:39:17.857-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:39:17.874-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:39:20.086-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:39:20.114-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:39:20.493-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:39:20.493-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:39:20.493-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:39:20.493-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:20.493-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:20.541-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:20.541-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:20.564-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:39:20.833-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:20.850-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:39:21.101-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:21.116-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:39:21.408-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:21.408-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:21.408-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:21.408-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:21.408-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:21.485-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:21.485-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:21.498-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:39:21.877-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:39:21.894-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:39:22.182-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:39:22.206-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:39:22.810-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:39:22.810-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:39:22.810-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:39:22.810-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:22.810-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:22.870-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:22.870-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:22.891-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
CapiNozzle 2017-04-25T13:39:23.247-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:39:23.262-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
CapiNozzle 2017-04-25T13:39:23.603-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:39:23.620-05:00 [INFO] Retrying update docs for vb 105 for the 3th time
CapiNozzle 2017-04-25T13:39:24.059-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:39:24.059-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 105 after 3 retries
CapiNozzle 2017-04-25T13:39:24.059-05:00 [ERROR] Raise error condition batch update docs failed for vb 105 after 3 retries
GenericSupervisor 2017-04-25T13:39:24.059-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:24.059-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:24.122-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:24.122-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:24.133-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
GenericSupervisor 2017-04-25T13:39:24.419-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:39:24.419-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:39:24.419-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:39:24.475-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:24.486-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
ToplogyChangeDetector 2017-04-25T13:39:24.625-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:39:25.064-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:25.074-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:39:25.180-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:39:25.191-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:39:25.396-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:25.407-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:25.407-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:39:25.407-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:39:25.407-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
GenericSupervisor 2017-04-25T13:39:25.407-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:25.407-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:25.486-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:25.486-05:00 [ERROR] batchUpdateDocs for vb 109 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:25.497-05:00 [INFO] Retrying update docs for vb 109 for the 1th time
CapiNozzle 2017-04-25T13:39:25.510-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:25.522-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:39:25.611-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:25.611-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:39:25.611-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:39:25.611-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:25.611-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:25.732-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:25.732-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:25.743-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:39:25.911-05:00 [ERROR] batchUpdateDocs for vb 109 failed with err receieved unexpected status code, 500, from update docs request for vb 109
.
CapiNozzle 2017-04-25T13:39:25.922-05:00 [INFO] Retrying update docs for vb 109 for the 2th time
PipelineManager 2017-04-25T13:39:25.995-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:39:25.995-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:39:25.995-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:25.995-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
PipelineRuntimeCtx 2017-04-25T13:39:25.995-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:39:25.995-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:39:25.995-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:39:25.995-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:39:25.995-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:25.996-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:39:25.996-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:25.996024112 -0500 CDT
GenericSupervisor 2017-04-25T13:39:25.996-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:25.996-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:39:25.996-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:25.996-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:39:25.996-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:25.996-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:39:25.996-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-331919349 has been stopped
GenericPipeline 2017-04-25T13:39:25.996-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-331919349 is stopped
PipelineManager 2017-04-25T13:39:25.996-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
PipelineManager 2017-04-25T13:39:25.996-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:15.995304592-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:39:25.996-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
PipelineManager 2017-04-25T13:39:25.996-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:25.996-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
GenericPipeline 2017-04-25T13:39:25.996-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
GenericPipeline 2017-04-25T13:39:25.996-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-331919349
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:39:25.996-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:39:25.996-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:25.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:39:25.996-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:39:25.997-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:39:25.997-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:25.997-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:25.997-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:25.997-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:39:25.997-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:39:25.997-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:39:25.997-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:25.997310696 -0500 CDT
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:39:25.997-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:39:25.997-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:25.997598621 -0500 CDT
AsyncListener 2017-04-25T13:39:25.997-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:39:25.998-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:39:25.998-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:25.998-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:39:25.998-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:39:25.998-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:25.998-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:39:25.998-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:39:25.998-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:25.998-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:39:25.998-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:25.998-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:39:26.000-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:26.000-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:26.000-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:26.000312399 -0500 CDT
DcpNozzle 2017-04-25T13:39:26.000-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:26.000324086 -0500 CDT
DcpNozzle 2017-04-25T13:39:26.000-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:39:26.000-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:39:26.000-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:26.000-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:26.000-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:39:26.000-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:39:26 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:39:26 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:39:26 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:39:26 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:39:26 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:39:26.116-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:26.117-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:39:26.117-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:39:26.175-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:26.176-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:39:26.176-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:26.176-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:39:26.176-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:26.176-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:39:26.176-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:39:26.177-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:39:26.177-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
GenericSupervisor 2017-04-25T13:39:26.179-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:39:26.179-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
GenericSupervisor 2017-04-25T13:39:26.179-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:26.179-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:26.179-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:26.179-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:26.179-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:39:26.179-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
CheckpointManager 2017-04-25T13:39:26.179-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
CapiUtils 2017-04-25T13:39:26.208-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:26.208-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:26.208-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:39:26.209-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
StatisticsManager 2017-04-25T13:39:26.210-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:39:26.210-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
XDCRFactory 2017-04-25T13:39:26.210-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:39:26.210-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346 is constructed, start it
GenericPipeline 2017-04-25T13:39:26.211-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[24 44 87 91 139 6 60 36 118 7 94 79 115 150 27 119 4 149 43 42 77 104 138 83 88 125 62 69 96 114 155 162 113 161 12 40 92 98 11 35 51 135 54 166 100 65 154 10 152 3 67 89 105 32 56 71 103 97 133 144 47 108 14 31 64 102 111 130 2 19 73 74 123 34 58 30 49 37 117 170 164 26 39 120 131 137 146 160 46 61 78 82 110 126 48 76 143 20 95 147 109 142 80 122 70 66 107 18 167 13 128 140 153 159 165 21 141 5 29 68 112 45 145 148 156 157 22 86 116 81 93 168 15 17 23 28 99 121 158 25 33 63 129 132 0 16 75 90 101 84 72 1 38 134 9 59 163 50 52 57 85 124 136 41 106 151 169 8 53 55 127] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[297 284 296 317 233 178 187 224 181 295 299 293 322 218 227 310 200 202 203 272 316 327 279 338 223 274 172 194 318 309 331 302 189 221 240 269 261 282 286 329 217 334 335 186 222 237 226 324 337 175 314 215 290 307 323 270 275 326 195 340 328 333 198 211 256 292 174 229 241 247 288 321 231 235 179 205 210 238 252 313 191 201 301 339 281 291 273 300 176 230 234 320 330 259 185 209 213 306 183 190 216 251 184 188 212 276 192 207 280 250 271 220 287 244 196 248 311 171 208 232 243 242 277 219 249 268 182 258 303 245 246 315 236 254 304 255 325 283 180 214 263 193 199 239 253 257 278 204 197 262 289 206 294 332 260 285 319 267 177 225 265 266 305 264 173 228 312 336 298 308]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[92 98 12 40 51 135 11 35 54 166 100 65 154 10 152 89 105 3 67 71 103 32 56 97 133 144 47 108 64 102 111 130 14 31 2 19 123 73 74 34 58 30 49 170 37 117 120 131 137 146 164 26 39 78 82 110 126 160 46 61 143 48 76 20 95 147 109 142 80 122 70 66 107 18 140 153 159 165 167 13 128 21 141 5 112 29 68 45 145 157 148 156 116 22 86 168 81 93 23 28 99 121 15 17 158 63 129 132 25 33 75 90 101 0 16 84 72 134 1 38 163 9 59 57 85 124 136 50 52 151 169 41 106 55 127 8 53 87 91 139 24 44 6 60 36 118 7 94 150 79 115 27 119 4 149 43 104 138 42 77 125 83 88 96 114 155 162 62 69 113 161] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[172 194 223 274 318 309 331 189 221 240 269 302 286 329 261 282 334 335 217 186 222 237 337 226 324 175 314 215 290 270 275 307 323 326 340 195 198 211 256 292 328 333 174 229 288 241 247 321 179 205 210 231 235 313 238 252 191 201 301 339 281 291 176 230 234 273 300 320 330 185 209 213 259 183 190 216 251 306 188 212 276 184 192 207 280 250 271 220 287 244 248 311 196 208 232 243 171 242 277 268 219 249 182 258 303 246 245 236 254 304 315 255 325 180 214 263 283 193 199 239 253 278 257 197 204 206 262 289 294 332 285 319 260 177 225 265 267 266 305 173 228 264 298 308 312 336 297 284 296 317 233 178 187 224 181 295 299 293 322 218 227 310 200 202 316 327 203 272 279 338]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 stats_interval:1000 active:true checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: doc_batch_size_kb:2048 failure_restart_interval:10]
CheckpointManager 2017-04-25T13:39:26.211-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346...
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:26.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
CheckpointManager 2017-04-25T13:39:26.219-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:39:26.219-05:00 [INFO] Checkpointing starts in 1802.617 sec
PipelineRuntimeCtx 2017-04-25T13:39:26.219-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:39:26.219-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:39:26.220-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:39:26.228-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:39:26.228-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:39:26.228-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:39:26.228-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:26.228-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:39:26.228-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:39:26.228-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:39:26.230-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:39:26.234-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:39:26.249-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:26.249-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:26.249-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:39:26.253-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:39:26.276-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:39:26.277-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:39:26.277-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CheckpointManager 2017-04-25T13:39:26.285-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:39:26.310-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:39:26.310-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:39:26.310-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:39:26.361-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:39:26.361-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:26.362-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:39:26.362-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:39:26.362-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:39:26.362-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:39:26.379-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:39:26.379-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:39:26.379-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:39:26.392-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:39:26.392-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:39:26.436-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:39:26.545-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:39:26.545-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:26.545-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:39:26.545-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:39:26.546-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:39:26.546-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:39:26.546-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346 is started----------
PipelineManager 2017-04-25T13:39:26.546-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:39:26.546-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:39:26.546-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:39:27.287-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:39:27.621-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:39:27.622-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:39:27.622-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:39:27.622-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:39:27 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:39:27.782-05:00 [INFO] Send batch count=49
CapiNozzle 2017-04-25T13:39:27.797-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:27.797-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:39:27 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:39:27.819-05:00 [INFO] Send batch count=172
CapiNozzle 2017-04-25T13:39:27.871-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:39:27 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 308 successful
CapiNozzle 2017-04-25T13:39:27.947-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:27.947-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:39:27 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:39:27 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:39:27.983-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
2017/04/25 13:39:28 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 8 successful
CapiNozzle 2017-04-25T13:39:28.022-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:39:28.058-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:39:28 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 129 successful
CapiNozzle 2017-04-25T13:39:28.201-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:39:28.231-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
2017/04/25 13:39:28 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 133 successful
StatisticsManager 2017-04-25T13:39:28.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:39:28 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 155 successful
CapiNozzle 2017-04-25T13:39:28.607-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:39:28.626-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
2017/04/25 13:39:28 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 165 successful
CapiNozzle 2017-04-25T13:39:28.670-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:39:28.670-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:39:28.670-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:39:28.670-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:39:28.670-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
PipelineManager 2017-04-25T13:39:28.678-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:39:28.679-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:39:28.679-05:00 [INFO] err_list=[{"time":"2017-04-25T13:39:28.679337538-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:39:28.679-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:39:28.770-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:28.770-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:39:28 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 166 successful
CapiNozzle 2017-04-25T13:39:28.790-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
PipelineManager 2017-04-25T13:39:28.833-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:28.679337538-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]}
]
2017/04/25 13:39:28 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:39:28 UPR_STREAMREQ for vb 174 successful
CapiNozzle 2017-04-25T13:39:29.041-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:39:29.054-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
2017/04/25 13:39:29 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 93 successful
CheckpointManager 2017-04-25T13:39:29.176-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
CheckpointManager 2017-04-25T13:39:29.176-05:00 [INFO] Done with SetVBTimestamps
StatisticsManager 2017-04-25T13:39:29.238-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:39:29.239-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346 {"Errors": "[{\"time\":\"2017-04-25T13:39:28.679337538-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 20, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 80, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 12801, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 80, "set_filtered": 0, "set_received_from_dcp": 12806, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:39:29.248-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:39:29 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:39:29 UPR_STREAMREQ for vb 197 successful
CapiNozzle 2017-04-25T13:39:29.272-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
DcpNozzle 2017-04-25T13:39:29.274-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:39:29.352-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:39:29.352-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:39:29.352-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:39:29.352-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:29.352-05:00 [INFO] Send batch count=500
DcpNozzle 2017-04-25T13:39:29.358-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:39:29.438-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:29.439-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:29.457-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:39:29.735-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:39:29.756-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:39:29.817-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:39:29.835-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:39:30.135-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:39:30.135-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:39:30.135-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:39:30.135-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:30.135-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:30.244-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:39:30.249-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:30.249-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:30.263-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:39:30.266-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:39:30.566-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:39:30.566-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:39:30.566-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:39:30.566-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:30.566-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:30.657-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:30.657-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:30.671-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:39:30.860-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:39:30.898-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:39:31.829-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:39:31.842-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:39:32.076-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:39:32.105-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:39:32.290-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:39:32.305-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:39:32.454-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:39:32.454-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:39:32.454-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:39:32.454-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:32.454-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:32.533-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:32.533-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:32.547-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:39:32.675-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:39:32.675-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:39:32.675-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:39:32.675-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:32.675-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:32.755-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:32.757-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:32.771-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:39:32.957-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:32.969-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:39:33.148-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:39:33.171-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:39:33.412-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:33.428-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:39:33.567-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:39:33.582-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:39:33.937-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:33.937-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:39:33.937-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:39:33.937-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:33.937-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:34.031-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:34.031-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:34.046-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:39:34.156-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:39:34.156-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:39:34.156-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:39:34.156-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:34.156-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:34.240-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:34.241-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:34.262-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:39:34.359-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:39:34.370-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:39:34.640-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:39:34.656-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:39:34.778-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:39:34.797-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:39:35.625-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:39:35.636-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:39:35.754-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:39:35.754-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:39:35.754-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:39:35.754-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:35.754-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:35.856-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:35.857-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:35.867-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:39:36.006-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:39:36.006-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:39:36.006-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:39:36.006-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:36.006-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:36.109-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:36.109-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:36.120-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
GenericSupervisor 2017-04-25T13:39:36.230-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:39:36.231-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:39:36.231-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:39:36.279-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:39:36.290-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
ToplogyChangeDetector 2017-04-25T13:39:36.393-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:39:36.608-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:39:36.619-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:39:36.814-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:39:36.825-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:39:36.987-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:39:36.998-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:39:37.172-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:39:37.173-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:39:37.173-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:39:37.173-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:37.173-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:37.247-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:37.248-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:37.258-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:39:37.403-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:39:37.403-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:39:37.403-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:39:37.403-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:37.403-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:37.501-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:37.501-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:37.512-05:00 [INFO] Retrying update docs for vb 307 for the 1th time
CapiNozzle 2017-04-25T13:39:37.648-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:37.659-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:39:37.877-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:39:37.888-05:00 [INFO] Retrying update docs for vb 307 for the 2th time
CapiNozzle 2017-04-25T13:39:38.037-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:38.047-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:39:38.272-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:39:38.284-05:00 [INFO] Retrying update docs for vb 307 for the 3th time
CapiNozzle 2017-04-25T13:39:38.491-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:38.491-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:38.491-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:38.491-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:38.491-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:38.579-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:38.579-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:38.590-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:39:38.651-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:39:38.651-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 307 after 3 retries
CapiNozzle 2017-04-25T13:39:38.652-05:00 [ERROR] Raise error condition batch update docs failed for vb 307 after 3 retries
GenericSupervisor 2017-04-25T13:39:38.652-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:38.652-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:38.669-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 310
PipelineManager 2017-04-25T13:39:38.680-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:39:38.680-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:39:38.680-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:38.680-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
PipelineRuntimeCtx 2017-04-25T13:39:38.680-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:39:38.680-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:39:38.680-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:38.680-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:39:38.680-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:38.680418907 -0500 CDT
GenericSupervisor 2017-04-25T13:39:38.680-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:38.680-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:39:38.680-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:38.680-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:39:38.680-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:38.680-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:39:38.680-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:39:38.680-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:39:38.680-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-176967346 has been stopped
GenericPipeline 2017-04-25T13:39:38.680-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-176967346 is stopped
PipelineManager 2017-04-25T13:39:38.680-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:39:38.681-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:28.679337538-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:39:38.681-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:38.681-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
GenericPipeline 2017-04-25T13:39:38.681-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
GenericPipeline 2017-04-25T13:39:38.681-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
GenericPipeline 2017-04-25T13:39:38.681-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-176967346
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:38.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:38.682-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:39:38.682-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:38.682-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:39:38.682-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:38.682-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:39:38.682-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:38.682-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:39:38.682-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:38.682-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:38.682-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:39:38.683-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:38.682486277 -0500 CDT
DcpNozzle 2017-04-25T13:39:38.683-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:39:38.683-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:38.682806508 -0500 CDT
CapiNozzle 2017-04-25T13:39:38.683-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:39:38.683-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:38.683-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:38.683-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:39:38.683-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.683-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:38.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:39:38.684-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:38.684-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:39:38.698-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:38.698-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:38.698-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:38.698063994 -0500 CDT
DcpNozzle 2017-04-25T13:39:38.698-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:38.698082524 -0500 CDT
DcpNozzle 2017-04-25T13:39:38.698-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:39:38.698-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:39:38.698-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:38.698-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:39:38.698-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:38.698-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:39:38 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:39:38 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:39:38.797-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:39:38.797-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:39:38.797-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:38.797-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:39:38.806-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:38.806-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:39:38.806-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:39:38.915-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:39:38.915-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:39:38.915-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:38.915-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:39:38.916-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:39:38.916-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:39:38.916-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:39:38.916-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:39:38.916-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
GenericSupervisor 2017-04-25T13:39:38.918-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:39:38.918-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
GenericSupervisor 2017-04-25T13:39:38.918-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:38.918-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:38.918-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:38.918-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:39:38.918-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:39:38.918-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
CheckpointManager 2017-04-25T13:39:38.918-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
CapiUtils 2017-04-25T13:39:38.973-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:38.973-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:38.973-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:39:38.974-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
StatisticsManager 2017-04-25T13:39:38.974-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:39:38.974-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
XDCRFactory 2017-04-25T13:39:38.975-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:39:38.975-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310 is constructed, start it
GenericPipeline 2017-04-25T13:39:38.976-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[232 233 285 292 317 321 282 288 318 330 186 339 205 266 336 256 240 277 279 284 268 278 305 213 260 312 193 306 174 209 298 324 325 196 281 286 252 248 212 289 310 207 247 307 332 340 326 217 224 273 314 322 250 255 173 176 206 265 262 313 290 294 211 239 264 299 320 221 329 272 175 245 210 226 271 283 257 251 215 241 218 242 293 338 182 287 192 237 199 204 190 191 179 194 203 220 172 327 259 309 178 225 228 291 308 319 180 202 236 198 185 238 269 337 258 263 171 200 229 296 304 323 335 301 222 261 246 253 276 243 275 303 230 274 331 227 231 270 316 177 328 183 197 214 234 267 334 223 297 184 219 295 181 216 254 208 235 188 315 201 249 300 195 302 244 333 189 280 311 187] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[17 27 75 104 7 9 35 123 4 83 135 160 166 44 122 109 69 144 153 22 40 48 128 74 96 133 164 18 88 106 24 54 105 108 66 25 32 86 30 73 84 139 10 29 81 99 38 149 155 170 41 20 23 78 5 159 60 8 124 49 72 93 168 51 116 148 151 6 87 101 113 140 152 14 118 136 143 58 119 120 137 89 91 16 70 167 169 92 95 126 142 100 117 63 114 97 103 115 163 146 162 3 57 121 130 134 62 80 59 79 52 65 145 1 12 110 157 31 125 2 42 50 161 13 47 107 11 19 28 56 90 111 112 150 132 34 45 67 68 26 77 94 138 0 53 156 71 21 36 43 61 64 147 158 85 102 129 15 55 76 141 154 46 98 33 37 39 131 82 127 165]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[16 70 167 169 92 95 126 142 100 117 63 114 97 103 115 163 3 57 121 130 146 162 134 62 80 59 79 52 65 145 1 12 110 157 31 125 2 42 50 161 13 47 11 19 28 56 107 90 111 112 150 34 45 67 68 132 26 77 94 138 0 53 156 21 36 43 61 71 64 147 158 85 102 129 15 55 76 141 154 46 98 33 37 39 131 82 127 165 17 27 75 104 7 9 35 123 4 83 135 160 166 44 122 109 69 144 153 22 40 48 128 74 96 133 164 18 88 106 24 54 105 108 66 25 32 86 30 73 84 139 10 29 81 99 38 149 155 170 41 20 23 78 5 159 60 8 124 49 72 93 51 116 148 151 168 6 87 101 113 140 152 14 118 136 143 58 119 120 137 89 91] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[257 215 241 251 218 242 293 338 182 287 192 237 190 191 199 204 194 179 203 220 172 259 309 327 178 225 319 180 202 228 291 308 198 236 337 185 238 269 171 200 258 263 323 229 296 304 335 222 261 301 276 246 253 243 275 303 230 274 331 316 177 227 231 270 328 214 183 197 234 267 334 184 223 297 181 219 295 216 254 188 208 235 201 249 300 315 195 302 244 333 189 280 311 187 292 232 233 285 317 321 330 186 282 288 318 205 266 336 339 256 240 277 279 284 213 268 278 305 260 312 193 306 174 325 209 298 324 196 281 286 248 252 212 289 310 207 247 307 332 340 217 224 326 250 255 273 314 322 173 176 206 265 262 313 211 239 290 294 221 264 299 320 329 175 245 272 210 226 271 283]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: failure_restart_interval:10 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:39:38.976-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310...
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:39:38.988-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:39:38.988-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:38.988-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:39:38.988-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:39:38.988-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:39:38.988-05:00 [INFO] Checkpointing starts in 1804.053 sec
PipelineRuntimeCtx 2017-04-25T13:39:38.989-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:39:38.989-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:39:38.989-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:39:38.989-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:39:38.991-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:39:38.991-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:39:38.991-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:39:38.991-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:39:38.995-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:39:39.021-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:39:39.021-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:39:39.021-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:39:39.051-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:39:39.051-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CapiUtils 2017-04-25T13:39:39.054-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:39:39.054-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:39:39.054-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:39:39.069-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:39:39.073-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:39:39.074-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:39:39.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:39:39.091-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:39.091-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:39:39.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:39:39.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:39:39.091-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:39:39.122-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:39:39.122-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:39:39.122-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:39:39.139-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:39:39.140-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:39:39.215-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:39:39.347-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:39:39.347-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:39:39.347-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:39:39.377-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:39:39.377-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:39:39.377-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:39:39.377-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310 is started----------
PipelineManager 2017-04-25T13:39:39.377-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:39:39.377-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:39:39.377-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:39:40.106-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:39:41.091-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:39:41.091-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:39:41.091-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:39:41.091-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
StatisticsManager 2017-04-25T13:39:41.110-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:39:41 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:39:41.191-05:00 [INFO] Send batch count=205
2017/04/25 13:39:41 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:39:41.239-05:00 [INFO] Send batch count=166
CapiNozzle 2017-04-25T13:39:41.281-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:41.281-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:41.294-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:41.295-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:39:41 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:39:41.332-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:39:41 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:39:41.345-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:39:41 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 46 successful
CapiNozzle 2017-04-25T13:39:41.688-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:41.714-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:39:41 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 149 successful
CapiNozzle 2017-04-25T13:39:41.837-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:41.876-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:39:41 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 262 successful
CapiNozzle 2017-04-25T13:39:41.967-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:39:41 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 166 successful
CapiNozzle 2017-04-25T13:39:41.981-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
2017/04/25 13:39:41 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:39:41 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 265 successful
StatisticsManager 2017-04-25T13:39:42.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:39:42 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 298 successful
CheckpointManager 2017-04-25T13:39:42.418-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
CheckpointManager 2017-04-25T13:39:42.418-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:39:42.483-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:39:42 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:39:42 UPR_STREAMREQ for vb 198 successful
DcpNozzle 2017-04-25T13:39:42.598-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:39:42.603-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:42.620-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:39:42.945-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:42.945-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:39:42.945-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:39:42.945-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]
ReplicationManager 2017-04-25T13:39:42.945-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]
PipelineManager 2017-04-25T13:39:42.952-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:39:42.952-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:39:42.957-05:00 [INFO] err_list=[{"time":"2017-04-25T13:39:42.957938301-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]
PipelineManager 2017-04-25T13:39:42.958-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:39:43.005-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:39:43.006-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310 {"Errors": "[{\"time\":\"2017-04-25T13:39:42.957938301-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 15, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 157, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 33507, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 157, "set_filtered": 0, "set_received_from_dcp": 33507, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:39:43.052-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:43.052-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:43.073-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:39:43.213-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:43.227-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:43.227-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:43.227-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:43.228-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:43.376-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:43.376-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:43.390-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:39:43.448-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:43.471-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:39:43.732-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:43.755-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
PipelineManager 2017-04-25T13:39:43.826-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:42.957938301-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries]}
]
CapiNozzle 2017-04-25T13:39:43.853-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:43.871-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:39:44.065-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:44.087-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:39:44.214-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:39:44.214-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:39:44.214-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:39:44.214-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:44.214-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:44.320-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:44.320-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:44.353-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:39:44.430-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:39:44.430-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:39:44.430-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:39:44.430-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:39:44.430-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:39:44.970-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:39:45.006-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:39:45.007-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:39:45.034-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:39:45.070-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
GenericSupervisor 2017-04-25T13:39:49.621-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:39:49.587-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:39:49.672-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
GenericSupervisor 2017-04-25T13:39:49.699-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:39:49.654-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
GenericSupervisor 2017-04-25T13:39:50.727-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:39:50.858-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:39:50.941-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
PipelineManager 2017-04-25T13:39:53.501-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries 
PipelineManager 2017-04-25T13:39:53.952-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:39:54.007-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:54.011-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
CapiNozzle 2017-04-25T13:39:54.012-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:39:54.025-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:39:54.027-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:39:54.038-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
PipelineRuntimeCtx 2017-04-25T13:39:54.070-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:39:54.145-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:54.148-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=3
CapiNozzle 2017-04-25T13:39:54.161-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:39:54.166-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:39:54.229-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:54.147094835 -0500 CDT
GenericSupervisor 2017-04-25T13:39:54.357-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:39:54.357-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CapiNozzle 2017-04-25T13:39:54.366-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CheckpointManager 2017-04-25T13:39:54.386-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:54.390-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:39:54.391-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:39:54.391-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:39:54.427-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:39:54.428-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:39:54.430-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:39:54.430-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-916151310 has been stopped
GenericPipeline 2017-04-25T13:39:54.509-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-916151310 is stopped
PipelineManager 2017-04-25T13:39:54.509-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:39:54.521-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
GenericPipeline 2017-04-25T13:39:54.524-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
GenericPipeline 2017-04-25T13:39:54.524-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
GenericPipeline 2017-04-25T13:39:54.524-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-916151310
AsyncListener 2017-04-25T13:39:54.525-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:54.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:54.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:54.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:54.553-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:39:54.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:39:54.575-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:39:54.509-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:42.957938301-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:39:54.600-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:39:54.602-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:54.602-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:39:54.608-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:39:54.608-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:39:54.608-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:39:54.608-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:39:54.627-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:54.627-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:39:54.627-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:39:54.627-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:39:54.629-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:54.608193029 -0500 CDT
DcpNozzle 2017-04-25T13:39:54.629-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:39:54.629-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:54.627546557 -0500 CDT
CapiNozzle 2017-04-25T13:39:54.629-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:39:54.629-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:39:54.629-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:54.632-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:39:54.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:39:54.578-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:39:54.681-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:39:54.683-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:39:54.706-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:39:54.750-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:39:54.750-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:54.752-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:39:54.773-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:54.773-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:54.773259622 -0500 CDT
DcpNozzle 2017-04-25T13:39:54.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:39:54.773-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:54.773-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:39:54.773-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:39:54.773-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:39:54.773951269 -0500 CDT
DcpNozzle 2017-04-25T13:39:54.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:39:54.774-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:54.774-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:39:54.785-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:39:54.785-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:39:54.785-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:39:54.785-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:39:54.790-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
GenericSupervisor 2017-04-25T13:40:02.076-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:40:03.488-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:40:03.505-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:40:03.728-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:39:42.957938301-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:40:03.779-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:40:03.793-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:40:03.795-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:40:03.857-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:40:03.864-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:40:03.873-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:40:03.873-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:40:03.873-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:40:03.873-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:40:03.874-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:40:03.876-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:40:03.887-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
GenericSupervisor 2017-04-25T13:40:03.899-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:40:03.900-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
GenericSupervisor 2017-04-25T13:40:03.901-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:03.901-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:03.901-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:03.901-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:03.901-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:40:03.907-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
CheckpointManager 2017-04-25T13:40:03.908-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
ReplicationManager 2017-04-25T13:40:03.908-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
CapiUtils 2017-04-25T13:40:03.939-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:40:03.941-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:40:03.942-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:40:03.947-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
StatisticsManager 2017-04-25T13:40:03.961-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:40:03.962-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
XDCRFactory 2017-04-25T13:40:03.963-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:40:03.964-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 is constructed, start it
GenericPipeline 2017-04-25T13:40:03.972-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[154 34 47 14 79 25 63 75 119 138 15 59 133 91 128 167 66 149 98 78 83 30 54 126 168 22 33 169 37 135 62 86 156 56 57 87 148 161 26 92 81 109 113 155 42 108 20 61 141 16 70 39 2 6 115 131 44 1 17 80 100 137 38 55 150 64 114 65 107 111 117 134 159 43 53 106 28 48 9 101 163 7 90 24 52 122 147 0 10 27 132 49 84 146 116 121 18 89 153 76 136 124 31 68 145 50 51 23 77 12 58 125 160 36 123 72 139 85 140 41 67 74 94 69 93 104 46 60 105 158 4 157 82 88 97 99 110 120 165 5 29 13 35 164 112 162 96 144 3 8 130 32 45 19 21 40 142 102 170 143 151 71 103 129 152 166 73 127 118 11 95] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[320 327 182 204 260 269 265 173 200 325 280 315 331 290 242 176 209 273 330 340 191 225 262 307 339 184 211 245 275 285 207 237 288 279 213 258 195 210 266 287 310 270 314 248 249 218 231 253 272 291 261 236 254 289 193 286 175 177 227 267 304 323 281 294 308 171 300 317 293 252 299 311 174 243 234 283 312 306 332 217 221 263 274 284 187 196 268 309 324 203 216 334 189 186 224 318 333 185 319 336 180 229 276 321 335 337 172 259 277 282 292 326 233 238 250 222 235 197 296 302 228 298 303 338 206 199 246 241 223 232 214 240 201 264 329 181 194 244 316 178 192 212 215 251 271 278 179 205 219 183 297 301 198 208 220 239 188 226 247 255 256 305 313 328 190 257 295 202 230 322]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[33 54 126 168 22 135 169 37 57 62 86 156 56 87 92 148 161 26 109 81 155 113 42 61 108 20 70 141 16 6 39 2 131 115 17 44 1 55 80 100 137 38 114 150 64 53 65 107 111 117 134 159 43 48 106 28 101 9 90 163 7 24 10 52 122 147 0 132 27 84 49 121 146 116 18 89 136 153 76 68 124 31 51 145 50 77 23 58 12 123 125 160 36 139 72 67 85 140 41 74 93 94 69 60 104 46 105 157 158 4 82 29 88 97 99 110 120 165 5 35 13 162 164 112 8 96 144 3 45 130 32 21 19 142 40 170 102 151 143 103 71 127 129 152 166 73 95 118 11 47 154 34 79 14 25 59 63 75 119 138 15 128 133 91 149 167 66 83 98 78 30] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[176 209 273 330 340 191 184 225 262 307 339 211 245 275 285 207 237 288 258 279 213 195 210 266 287 310 248 270 314 231 249 218 253 272 291 261 236 254 289 193 286 175 177 227 267 304 323 281 294 308 171 300 317 293 252 299 311 174 243 234 312 283 217 306 332 221 263 274 284 187 196 268 309 324 203 216 334 189 186 224 318 333 185 319 336 180 229 276 321 335 337 172 259 277 282 292 326 233 238 250 197 222 235 296 302 228 298 303 206 338 199 246 232 241 223 240 214 201 264 329 181 194 244 316 178 192 212 215 251 271 278 179 205 219 183 297 301 198 208 220 239 188 226 247 255 256 305 328 313 190 257 295 202 230 322 320 327 182 204 260 269 265 173 200 325 280 315 331 242 290]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi worker_batch_size:500 source_nozzle_per_node:2 log_level:Info target_nozzle_per_node:2 stats_interval:1000 filter_expression: active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256]
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:03.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:03.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:03.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:03.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:40:03.984-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:40:03.984-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:40:03.985-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CheckpointManager 2017-04-25T13:40:03.985-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047...
GenericSupervisor 2017-04-25T13:40:03.986-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:40:03.987-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:40:03.987-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:40:03.988-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:40:03.993-05:00 [INFO] Checkpointing starts in 1802.103 sec
PipelineRuntimeCtx 2017-04-25T13:40:03.993-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:40:03.993-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:40:03.993-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:40:03.997-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:40:04.001-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:40:04.004-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:40:04.011-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
ToplogyChangeDetector 2017-04-25T13:40:04.036-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:40:04.051-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:40:04.052-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:40:04.054-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:40:04.061-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:40:04.073-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CapiUtils 2017-04-25T13:40:04.087-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:40:04.087-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:40:04.088-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:40:04.174-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:40:04.205-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:40:04.251-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:40:04.288-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:40:04.289-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:40:04.289-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:40:04.303-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:40:04.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:40:04.335-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:40:04.358-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:40:04.358-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:40:04.358-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:40:04.388-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:40:04.388-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:40:04.613-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:40:04.613-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:40:04.613-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:40:04.640-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:40:04.640-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:40:04.640-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:40:04.640-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 is started----------
PipelineManager 2017-04-25T13:40:04.642-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:40:04.642-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:40:04.643-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:40:05.084-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:40:05.411-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:40:05.421-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:40:05.423-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:40:05.425-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:40:05 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:40:05 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 26 successful
StatisticsManager 2017-04-25T13:40:06.068-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:40:06 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 147 successful
CapiNozzle 2017-04-25T13:40:06.647-05:00 [INFO] Send batch count=1
2017/04/25 13:40:06 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 52 successful
CapiNozzle 2017-04-25T13:40:06.668-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:40:06.676-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:40:06.676-05:00 [INFO] Send batch count=112
2017/04/25 13:40:06 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 59 successful
CapiNozzle 2017-04-25T13:40:06.774-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:40:06.777-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:40:06.792-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:40:06 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:40:06 UPR_STREAMREQ for vb 78 successful
CapiNozzle 2017-04-25T13:40:07.015-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:40:07 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 288 successful
CheckpointManager 2017-04-25T13:40:07.027-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
CheckpointManager 2017-04-25T13:40:07.027-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:40:07.028-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:40:07 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 99 successful
StatisticsManager 2017-04-25T13:40:07.118-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:40:07 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:40:07 UPR_STREAMREQ for vb 198 successful
DcpNozzle 2017-04-25T13:40:07.142-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:40:07.208-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:40:07.340-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:40:07.340-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:40:07.490-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:40:07.500-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
StatisticsManager 2017-04-25T13:40:11.991-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:40:14.741-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:40:14.799-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:40:14.772-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:40:15.085-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:40:15.813-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
GenericSupervisor 2017-04-25T13:40:15.882-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:40:15.948-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:40:15.885-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:40:16.720-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:40:16.904-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:16.979-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:40:17.033-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:40:17.036-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:40:17.064-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:39:42.957938301-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 201 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 1547, "dcp_dispatch_time": 1485, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 370, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 13354, "docs_rep_queue": 12441, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 8324, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 370, "set_filtered": 0, "set_received_from_dcp": 13354, "size_rep_queue": 16274654, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 34}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 317, "docs_opt_repd": 0, "docs_rep_queue": 15825, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 317, "size_rep_queue": 19924499, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 418, "max": 42, "mean": 40.53349282296651, "min": 22}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 53, "docs_opt_repd": 0, "docs_rep_queue": 13639, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 53, "size_rep_queue": 16670699, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 99, "max": 10, "mean": 9.909090909090908, "min": 5}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 6991, "max": 1626818, "mean": 1943.253, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 17022, "expiry_received_from_dcp": 0, "set_received_from_dcp": 17022}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 206, "dcp_dispatch_time": {"count": 6361, "max": 893977, "mean": 983.425, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 14832, "expiry_received_from_dcp": 0, "set_received_from_dcp": 14832}}

CapiNozzle 2017-04-25T13:40:18.126-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:40:17.954-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 18081 items, sent 501 items
StatisticsManager 2017-04-25T13:40:18.204-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 16742 items, sent 114 items
StatisticsManager 2017-04-25T13:40:18.211-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:18511]
StatisticsManager 2017-04-25T13:40:18.212-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 18512 items, sent 18511 items.
StatisticsManager 2017-04-25T13:40:18.212-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:16742]
StatisticsManager 2017-04-25T13:40:18.212-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 16743 items, sent 16742 items.
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =730 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:40:18.213-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =730 
StatisticsManager 2017-04-25T13:40:20.466-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:40:22.549-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:40:22.639-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:40:22.646-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:40:22.834-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:40:24.149-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
StatisticsManager 2017-04-25T13:40:24.067-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:40:23.831-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:40:24.270-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:40:24.315-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:40:24.319-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:40:24.337-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:40:24.372-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 827, "dcp_dispatch_time": 1260, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 370, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 35992, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 370, "set_filtered": 0, "set_received_from_dcp": 35992, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

GenericSupervisor 2017-04-25T13:40:24.319-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:40:24.550-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:40:24.701-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:40:24.843-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:40:24.843-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:40:24.843-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:40:24.848-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:40:25.004-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:40:25.026-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:40:25.104-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:40:25.053-05:00 [INFO] err_list=[{"time":"2017-04-25T13:40:25.053715922-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:40:25.205-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:40:25.402-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:40:25.408-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:40:25.402-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:40:25.409-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:40:25.430-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:40:25.053715922-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:40:25.443-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:40:25.470-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:40:26.055-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:40:26.081-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:40:26.418-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:40:26.454-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:40:26.643-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:40:26.671-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:40:27.110-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:40:27.131-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:40:28.939-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:40:29.908-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:40:29.908-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:40:29.758-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:40:29.958-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:40:29.958-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:40:29.910-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
GenericSupervisor 2017-04-25T13:40:29.958-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:40:30.090-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:40:30.129-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:40:30.994-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:40:31.008-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:40:30.971-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:40:31.043-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:40:31.098-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:40:31.130-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
PipelineManager 2017-04-25T13:40:31.311-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:40:25.053715922-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:40:31.701-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:40:31.719-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:40:35.370-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:40:36.544-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:40:35.596-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
PipelineManager 2017-04-25T13:40:37.066-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:40:37.181-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
PipelineRuntimeCtx 2017-04-25T13:40:37.979-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:40:38.674-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:40:38.168-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
GenericSupervisor 2017-04-25T13:40:38.762-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:40:38.795-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:40:38.753781495 -0500 CDT
GenericSupervisor 2017-04-25T13:40:39.324-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:40:40.602-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:40:40.851-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:40:40.853-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:40:40.855-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:40:40.855-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:40:40.861-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:40:40.872-05:00 [INFO] StatisticsManager Stopped
GenericSupervisor 2017-04-25T13:40:40.752-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:40:41.019-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
ToplogyChangeDetector 2017-04-25T13:40:40.973-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:40:41.027-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 has been stopped
GenericPipeline 2017-04-25T13:40:41.863-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
AsyncListener 2017-04-25T13:40:41.908-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:40:41.925-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:40:41.926-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:40:41.927-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.934-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.955-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:40:41.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:40:41.967-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:40:41.967-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:41.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:40:41.969-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
GenericPipeline 2017-04-25T13:40:41.969-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:40:41.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:40:41.874-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
GenericPipeline 2017-04-25T13:40:41.998-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:40:42.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:40:42.024-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:40:41.881-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-874397047
GenericPipeline 2017-04-25T13:40:42.024-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:40:42.024-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:40:42.031-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:40:41.907-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-874397047 is stopped
PipelineManager 2017-04-25T13:40:42.039-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:40:42.049-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:40:42.049-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:40:42.049-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:40:42.070-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:40:42.070-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:40:42.070-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:40:41.973-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:40:42.007-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
DcpNozzle 2017-04-25T13:40:42.024-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:40:42.055-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:40:42.032626478 -0500 CDT
DcpNozzle 2017-04-25T13:40:43.852-05:00 [INFO] Ask uprfeed to close
CapiNozzle 2017-04-25T13:40:48.794-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericSupervisor 2017-04-25T13:40:49.623-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:40:50.088-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 2 consecutive heart beats
GenericPipeline 2017-04-25T13:40:48.561-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:40:50.259-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:40:50.262-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1, err=Execution timed out, let it alone to die
GenericSupervisor 2017-04-25T13:40:47.021-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:40:50.276-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:40:49.443-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
GenericPipeline 2017-04-25T13:40:47.260-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:40:50.334-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:40:50.336-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:40:50.336-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:40:47.611-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:40:50.360-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:40:50.360-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:40:50.360-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:40:47.303-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:40:50.360-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:40:50.360-05:00 [INFO] Failed to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:40:50.360-05:00 [INFO] Source nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 failed to stop in time, left it alone to die.
GenericPipeline 2017-04-25T13:40:50.362-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 failed to stop in time, left it alone to die.
GenericSupervisor 2017-04-25T13:40:50.323-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
DcpNozzle 2017-04-25T13:40:50.407-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:40:50.407377898 -0500 CDT
GenericSupervisor 2017-04-25T13:40:50.437-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:40:50.344-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:40:50.314043916 -0500 CDT
DcpNozzle 2017-04-25T13:40:50.514-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:40:50.556-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
DcpNozzle 2017-04-25T13:40:50.557-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:40:50.557-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:40:50.557880361 -0500 CDT
DcpNozzle 2017-04-25T13:40:50.557-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
PipelineManager 2017-04-25T13:40:50.584-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:40:25.053715922-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

CapiNozzle 2017-04-25T13:40:50.623-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
PipelineManager 2017-04-25T13:40:50.624-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:40:50.642-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:40:50.651-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:40:50.653-05:00 [INFO] releasing capi client
PipelineManager 2017-04-25T13:40:51.215-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:40:25.053715922-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:40:51.709-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:40:51.720-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:40:51.721-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:40:51.875-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:40:51.887-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:40:51.891-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:40:51.891-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:40:51.891-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:40:51.891-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:40:51.894-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:40:51.902-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:40:51.909-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
GenericSupervisor 2017-04-25T13:40:51.923-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:40:51.923-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
GenericSupervisor 2017-04-25T13:40:51.924-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:51.924-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:51.924-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:51.924-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:40:51.924-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:40:51.929-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
CheckpointManager 2017-04-25T13:40:51.929-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
CapiUtils 2017-04-25T13:40:51.995-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:40:51.996-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:40:51.997-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:40:52.001-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
StatisticsManager 2017-04-25T13:40:52.012-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:40:52.014-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
XDCRFactory 2017-04-25T13:40:52.014-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:40:52.015-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273 is constructed, start it
GenericPipeline 2017-04-25T13:40:52.020-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[112 164 167 0 5 107 50 63 141 40 102 136 58 97 103 129 93 31 75 32 43 154 160 48 70 96 116 4 18 66 23 131 38 124 81 157 46 56 69 25 95 87 109 132 122 133 152 9 104 106 60 82 110 13 26 119 143 147 151 162 2 30 39 12 59 77 90 65 140 67 135 14 113 163 16 49 61 15 88 7 144 159 37 148 27 55 138 78 118 165 115 121 149 11 45 47 10 57 105 170 166 8 21 51 41 73 80 134 137 155 86 94 120 53 3 98 127 100 101 22 24 71 153 168 52 92 145 44 108 114 146 62 72 83 20 89 128 123 158 1 161 150 28 33 74 64 84 156 68 17 54 126 139 117 169 19 42 85 35 99 111 34 79 91 6 29 142 125 130 36 76] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[321 202 246 340 179 247 298 303 177 308 235 294 285 327 224 305 317 236 258 300 270 313 332 333 201 212 338 275 183 336 339 252 197 211 200 238 185 207 184 186 190 244 248 260 311 312 331 174 214 293 314 195 295 304 297 205 220 283 319 203 181 182 324 213 259 265 233 264 279 288 171 267 230 188 239 172 269 301 329 242 299 209 176 326 227 287 296 232 196 198 225 307 221 261 286 194 226 251 276 204 228 272 335 271 178 318 240 255 274 325 193 250 257 277 306 175 199 263 309 215 315 291 280 289 223 210 218 243 282 292 320 249 208 241 273 180 231 234 316 302 217 173 253 256 216 237 337 290 322 219 229 284 323 192 245 268 222 254 281 328 330 334 187 189 266 310 191 206 278 262]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[141 40 102 136 58 97 103 129 93 31 75 32 43 154 160 48 70 96 116 4 18 66 23 131 38 124 81 157 46 56 69 25 95 87 109 132 122 133 152 9 104 106 60 82 110 13 26 119 143 147 151 162 2 30 39 12 59 77 90 65 140 67 135 14 113 163 16 49 61 15 88 7 144 159 37 148 27 55 138 78 118 165 115 121 149 11 45 47 10 57 105 170 166 8 21 51 41 73 80 134 137 155 86 94 120 53 3 98 127 100 101 22 24 71 153 168 52 92 145 44 108 114 146 62 72 83 20 89 128 123 158 1 161 150 28 33 74 64 84 156 68 17 54 126 139 117 169 19 42 85 35 99 111 34 79 91 6 29 142 125 130 36 76 112 164 167 0 5 107 50 63] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[246 340 179 247 298 303 177 308 235 294 285 327 224 305 317 236 258 300 270 313 332 333 201 212 338 275 183 336 339 252 197 211 200 238 185 207 184 186 190 244 248 260 311 312 331 174 214 293 314 195 295 304 297 205 220 283 319 203 181 182 324 213 259 265 233 264 279 288 171 267 230 188 239 172 269 301 329 242 299 209 176 326 227 287 296 232 196 198 225 307 221 261 286 194 226 251 276 204 228 272 335 271 178 318 240 255 274 325 193 250 257 277 306 175 199 263 309 215 315 291 280 289 223 210 218 243 282 292 320 249 208 241 273 180 231 234 316 302 217 173 253 256 216 237 337 290 322 219 229 284 323 192 245 268 222 254 281 328 330 334 187 189 266 310 191 206 278 262 321 202]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 filter_expression: active:true target_nozzle_per_node:2 log_level:Info replication_type:capi]
CheckpointManager 2017-04-25T13:40:52.031-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273...
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:40:52.039-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
StatisticsManager 2017-04-25T13:40:52.043-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:40:52.072-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:40:52.073-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:40:52.073-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:40:52.073-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:40:52.074-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:40:52.074-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:40:52.075-05:00 [INFO] updateStats started
CheckpointManager 2017-04-25T13:40:52.075-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:40:52.080-05:00 [INFO] Checkpointing starts in 1801.479 sec
PipelineRuntimeCtx 2017-04-25T13:40:52.080-05:00 [INFO] Service CheckpointManager has been started
GenericSupervisor 2017-04-25T13:40:52.080-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:40:52.083-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:40:52.086-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:40:52.107-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:40:52.107-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:40:52.110-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:40:52.121-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:40:52.121-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:40:52.184-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:40:52.189-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:40:52.226-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:40:52.226-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:40:52.228-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:40:52.234-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:40:52.234-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:40:52.234-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:40:52.242-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:40:52.243-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:40:52.243-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:40:52.257-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:40:52.257-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:40:52.257-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:40:52.282-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:40:52.282-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:40:52.282-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:40:52.387-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:40:52.387-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:40:52.387-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:40:52.388-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:40:52.388-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:40:52.388-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:40:52.388-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273 is started----------
PipelineManager 2017-04-25T13:40:52.388-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:40:52.388-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:40:52.395-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:40:52.404-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:40:53.188-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:40:53.329-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:40:53.347-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:40:53.353-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:40:53.353-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:40:53 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:40:53.476-05:00 [INFO] Send batch count=282
CapiNozzle 2017-04-25T13:40:53.493-05:00 [INFO] Send batch count=4
2017/04/25 13:40:53 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:40:53.534-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:40:53.552-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:40:53 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 7 successful
CapiNozzle 2017-04-25T13:40:53.570-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:40:53.584-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:40:53 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:40:53.626-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:40:53.642-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
2017/04/25 13:40:53 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:40:53 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 118 successful
CapiNozzle 2017-04-25T13:40:54.445-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:40:54 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:40:54 UPR_STREAMREQ for vb 22 successful
CapiNozzle 2017-04-25T13:40:54.919-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:40:55.837-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
2017/04/25 13:40:55 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:40:57 UPR_STREAMREQ for vb 220 successful
CapiNozzle 2017-04-25T13:40:56.118-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:40:57.796-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:40:58 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:40:58 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:40:58 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:40:58 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:40:58 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:40:58 UPR_STREAMREQ for vb 23 successful
GenericSupervisor 2017-04-25T13:40:58.434-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
2017/04/25 13:40:59 UPR_STREAMREQ for vb 222 successful
ReplicationManager 2017-04-25T13:40:59.505-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
2017/04/25 13:41:00 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:41:01 UPR_STREAMREQ for vb 121 successful
StatisticsManager 2017-04-25T13:41:01.271-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:41:01 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:41:01 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:41:01 UPR_STREAMREQ for vb 223 successful
StatisticsManager 2017-04-25T13:41:01.773-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:41:02 UPR_STREAMREQ for vb 122 successful
GenericSupervisor 2017-04-25T13:41:02.304-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:41:02.452-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:41:02 UPR_STREAMREQ for vb 224 successful
PipelineManager 2017-04-25T13:41:02.520-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
StatisticsManager 2017-04-25T13:41:02.531-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 1000, "dcp_dispatch_time": 580, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 129, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 10736, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 129, "set_filtered": 0, "set_received_from_dcp": 9985, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

GenericSupervisor 2017-04-25T13:41:02.437-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:41:02.593-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:41:02 UPR_STREAMREQ for vb 25 successful
GenericSupervisor 2017-04-25T13:41:02.579-05:00 [INFO] Finish executing 0x681fb0
PipelineManager 2017-04-25T13:41:02.576-05:00 [INFO] err_list=[{"time":"2017-04-25T13:41:02.576362189-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]
PipelineManager 2017-04-25T13:41:02.638-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:41:02.657-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:41:02.746-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:41:02 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:41:02 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:41:02 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:41:02 UPR_STREAMREQ for vb 325 successful
CAPIService 2017-04-25T13:41:38.348-05:00 [INFO] Received error when making rest call. baseURL=10.202.98.33:9091, path=_pre_replicate, ret_err=Post http://10.202.98.33:9091/_pre_replicate: EOF, statusCode=0, num_retry=5
PipelineManager 2017-04-25T13:41:38.331-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries 
PipelineManager 2017-04-25T13:41:38.497-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:41:38.499-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:41:38.500-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
PipelineRuntimeCtx 2017-04-25T13:41:38.501-05:00 [INFO] Pipeline context is stopping...
2017/04/25 13:41:38 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:41:38 UPR_STREAMREQ for vb 123 successful
GenericSupervisor 2017-04-25T13:41:38.503-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:41:38.686-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:41:38.688-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:41:38.616994359 -0500 CDT
GenericSupervisor 2017-04-25T13:41:38.699-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:41:38.706-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:41:38.707-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:41:38.707-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:41:38.710-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:41:38.710-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:41:38.711-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:41:38.711-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-894551273 has been stopped
GenericPipeline 2017-04-25T13:41:38.729-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-894551273 is stopped
PipelineManager 2017-04-25T13:41:38.729-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:41:38.732-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
GenericPipeline 2017-04-25T13:41:38.733-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
GenericPipeline 2017-04-25T13:41:38.734-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
GenericPipeline 2017-04-25T13:41:38.734-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-894551273
AsyncListener 2017-04-25T13:41:38.735-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:41:38.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:41:38.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:41:38.738-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:41:38.740-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:41:38.740-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:41:38.740-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:41:38.740-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:41:38.741-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:41:38.742-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:41:38.742-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:41:38.743-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:41:38.744-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:41:38.744-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:41:38.750-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:41:38.756-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:41:38.759-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:41:38.761-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:41:38.761-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:41:38.761-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:41:38.761-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:41:38.761-05:00 [INFO] There is no active stream for vb=171
DcpNozzle 2017-04-25T13:41:38.761-05:00 [INFO] There is no active stream for vb=172
DcpNozzle 2017-04-25T13:41:38.761-05:00 [INFO] There is no active stream for vb=173
DcpNozzle 2017-04-25T13:41:38.761-05:00 [INFO] There is no active stream for vb=174
DcpNozzle 2017-04-25T13:41:38.762-05:00 [INFO] There is no active stream for vb=175
DcpNozzle 2017-04-25T13:41:38.762-05:00 [INFO] There is no active stream for vb=176
DcpNozzle 2017-04-25T13:41:38.762-05:00 [INFO] There is no active stream for vb=177
DcpNozzle 2017-04-25T13:41:38.762-05:00 [INFO] There is no active stream for vb=178
DcpNozzle 2017-04-25T13:41:38.762-05:00 [INFO] There is no active stream for vb=179
DcpNozzle 2017-04-25T13:41:38.762-05:00 [INFO] There is no active stream for vb=180
DcpNozzle 2017-04-25T13:41:38.763-05:00 [INFO] There is no active stream for vb=181
panic: send on closed channel

goroutine 3123 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc3cc43cfc0, 0x8edb60, 0xc21689adc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc3cc5b5ef0, 0x8ed920, 0xc3db69f760, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3cc2ee300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 3 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 3 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 3 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080d0000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc2080d8370)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080d0000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080de001, 0x5ff, 0x5ff, 0xc2080dc080, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080de001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080d4000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4000, 0x8f07a0, 0xc2080664a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc208066480, 0xc2080da5e0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bc00, 0x7f4c41c25c10, 0xc208066480, 0x0, 0x0, 0xc2080da5e0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bc00, 0x7f4c41c25c10, 0xc208066480, 0xc20807c280, 0xc20801f660, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bc00, 0x7f4c41c25c10, 0xc208066480)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b5f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b5f90, 0xc2080be480, 0x7f4c41c24398, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 19 [select, 3 minutes]:
net/http.(*persistConn).readLoop(0xc2081020b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 20 [select, 3 minutes]:
net/http.(*persistConn).writeLoop(0xc2081020b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 25 [syscall, 3 minutes]:
syscall.Syscall(0x0, 0x0, 0xc2080c3000, 0x1000, 0xc20803b1d0, 0xc208032320, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc2080c3000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc2080c3000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc2080c3000, 0x1000, 0x1000, 0xc208152ff0, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208066ba0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208066ba0, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 26 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc20807a380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 27 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc20807a390)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 28 [semacquire]:
sync.(*WaitGroup).Wait(0xc20808e7e0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:191 +0x428
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 29 [select, 3 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 30 [select, 3 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20808e5e0, 0x11, 0xc20814ffc0, 0xc208066780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20808e5e0, 0x11, 0xc20814ffc0, 0xc208066780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20808e5e0, 0x11, 0xc20814ffc0, 0xc208066780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2081529b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 31 [select, 3 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20802b800, 0xf, 0xc20814efc0, 0xc208066780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20802b800, 0xf, 0xc20814efc0, 0xc208066780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20802b800, 0xf, 0xc20814efc0, 0xc208066780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc208152b40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 32 [select]:
net/http.(*persistConn).roundTrip(0xc210f59ad0, 0xc21191c370, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc3db5fc4e0, 0xc334a28460, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3db5fc4e0, 0x7f4c41c23fe8, 0xe94600, 0x4c, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3db5fc4e0, 0x4c, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3db5fc4e0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3db5fc4e0, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3db5fc4e0, 0x7f4c41c25f78, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3db4cc460, 0xc3db4722c0, 0x34, 0x7f4c41c25f78, 0xc3db57efe0, 0x958760, 0xc3d92ae1b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc3db5564e0, 0xc3db4722c0, 0x34, 0x958760, 0xc3d92ae1b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Client).GetPool(0xc3db5564e0, 0x9cae60, 0x7, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:767 +0x339
github.com/couchbase/goxdcr/utils.RemotePool(0xc2080daa60, 0x11, 0xc2080d8680, 0xd, 0xc2080d8690, 0xc, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:98 +0x4c5
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).cacheRef(0xc208066360, 0xc3db00f8c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:512 +0x1d3
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).refresh(0xc208066360, 0xc3db00f8c0, 0xc20808e1a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:551 +0x154
github.com/couchbase/goxdcr/metadata_svc.(*RemoteClusterService).RemoteClusterByUuid(0xc208066360, 0xc20808e1a0, 0x20, 0x1, 0xc3daf37760, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/remote_cluster_service.go:178 +0x2a1
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080e42a0, 0xc3d6815480, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:480 +0x51d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080e42a0, 0xc3d6815480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc208010700)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 33 [chan send]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208066b40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:110 +0x737
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 13 [IO wait, 3 minutes]:
net.(*pollDesc).Wait(0xc2080e4450, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e4450, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e43f0, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc2080d9c38)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208104050, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc208104050, 0xc208102108, 0xc20810a000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080da8c0, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208106360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208106360, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208106360, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080e0840)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080e0840, 0xc2080dec00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080dc640, 0xc2080dec00, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080dc640, 0xc2080dec00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080dc6c0, 0xc2080dec00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc208170000, 0xc20815c0c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc208170000, 0x80f160, 0xc2080dc780, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 35 [select]:
net/http.(*persistConn).roundTrip(0xc208102210, 0xc212404f10, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc334d29380, 0xc218064ea0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc334d29380, 0x7f4c41c23fe8, 0xc20802c090, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc334d29380, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc334d29380, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc334d29380, 0xeb0d80, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/goxdcr/utils.doRestCall(0xc334d29380, 0x0, 0x818220, 0xc3d8a926c8, 0xeb0d80, 0xc20802b450, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc218301b80, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:173 +0x201
github.com/couchbase/goxdcr/utils.QueryRestApi(0xc218301b80, 0x15, 0x9e0430, 0xc, 0xc2080b9600, 0x9bf4c0, 0x3, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:134 +0x17a
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostInfo(0xc20801f440, 0xc21816ff50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:113 +0x139
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:157 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostName(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:177 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyHost(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:50 +0x4c
github.com/couchbase/goxdcr/replication_manager.GetReplicationInfos(0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:507 +0x304
github.com/couchbase/goxdcr/replication_manager.(*Adminport).doGetAllReplicationInfosRequest(0xc208066b40, 0xc3daa62ea0, 0xc21816ec30, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:361 +0x73
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc208066b40, 0xc3daa62ea0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:188 +0x7d9
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc208066b40, 0xc21191c290, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc21191c290, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208066b58)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 36 [IO wait]:
net.(*pollDesc).Wait(0xc208010c30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010c30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208010bd0, 0x0, 0x7f4c41c23df0, 0xc20c764950)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038148, 0xc208151e38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208038148, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208066f60, 0x7f4c41c26568, 0xc208038148, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 11 [select, 3 minutes]:
net/http.(*persistConn).readLoop(0xc20802e0b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 12 [select, 3 minutes]:
net/http.(*persistConn).writeLoop(0xc20802e0b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 37 [IO wait, 3 minutes]:
net.(*pollDesc).Wait(0xc208010d10, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010d10, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010cb0, 0xc208166000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc2080d9e68)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208104038, 0xc208166000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc208104038, 0xc20802e108, 0xc208166000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20808bb60, 0xc208166000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208106300)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208106300, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc208106300, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080bf650)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080bf650, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc20815c640, 0xc2080df200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc20815c640, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc20815c680, 0xc2080df200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080d4300, 0xc20815c1c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080d4300, 0x80f160, 0xc20815c740, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 57 [runnable]:
net.(*pollDesc).Wait(0xc2080108b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080108b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010850, 0xc20817c000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc3d94fb0f0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038058, 0xc20817c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc208038058, 0xc208102268, 0xc20817c000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801e560, 0xc20817c000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208066720)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208066720, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208102210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1335 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c86c0, 0xc8, 0xc2a5fb6800, 0xec, 0x100, 0xc263c4f800, 0xbb, 0x100, 0xc28c4fbaa0, 0xc28c4fb9e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1568 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d680, 0x13a, 0xc2fa810100, 0xec, 0x100, 0xc2fa538000, 0x1f4, 0x200, 0xc2fa80b560, 0xc2fa80b4a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1327 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c8480, 0x64, 0xc2a647f000, 0xec, 0x100, 0xc2bffc3800, 0xc9, 0x100, 0xc2b2101380, 0xc2b21012c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 48 [IO wait]:
net.(*pollDesc).Wait(0xc2080e5100, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080e5100, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080e50a0, 0xc2080d1000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc3aba2f3a0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2081041d8, 0xc2080d1000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc2081041d8, 0xc208102528, 0xc2080d1000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20808a2c0, 0xc2080d1000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208107680)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208107680, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2081024d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 65 [select]:
net/http.(*persistConn).writeLoop(0xc2081024d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 58 [select]:
net/http.(*persistConn).writeLoop(0xc208102210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 951 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0x12e, 0xc26d899400, 0xec, 0x100, 0xc2680a0800, 0xb3, 0x100, 0xc26dac8540, 0xc26dac8480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 273 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x66, 0xc208055c00, 0xec, 0x100, 0xc21e450000, 0xc8, 0x100, 0xc21e5eade0, 0xc21e5ead20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1227 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c86c0, 0xca, 0xc2ba810700, 0xec, 0x100, 0xc2ba6cd800, 0xbc, 0x100, 0xc2baa8fa40, 0xc2baa8f980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3349 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2124056a0, 0x1dcd6500, 0xc3cc569f10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5df360, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5df360)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3347 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc212405620, 0x1dcd6500, 0xc3cc569d80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5de690, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5de690)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3336 [select]:
net/http.(*persistConn).writeLoop(0xc210f59ad0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2154 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x66, 0xc26d898900, 0xec, 0x100, 0xc33d530800, 0xc8, 0x100, 0xc33e4189c0, 0xc33e418900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1009 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x4, 0xc2a647e500, 0xea, 0x100, 0xc2a624e800, 0xa2, 0x100, 0xc2a6608300, 0xc2a6608240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1566 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d440, 0x6a, 0xc208055f00, 0xec, 0x100, 0xc2f7ef6800, 0xcc, 0x100, 0xc2f8630240, 0xc2f8630180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3363 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457ccc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3359 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3cc43cfc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3cc43cfc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457c9c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3117 [runnable]:
encoding/json.(*decodeState).objectInterface(0xc334a1bb30, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x25a
encoding/json.(*decodeState).valueInterface(0xc334a1bb30, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc334a1bb30, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc334a1bb30, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc334a1bb30, 0xc2137a18f0)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).object(0xc334a1bb30, 0x833e80, 0xc2137a18f0, 0xd4)
	/usr/local/go/src/encoding/json/decode.go:497 +0x3c8
encoding/json.(*decodeState).value(0xc334a1bb30, 0x812be0, 0xc2137a18f0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).unmarshal(0xc334a1bb30, 0x812be0, 0xc2137a18f0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:149 +0x1e7
encoding/json.Unmarshal(0xc3d5e0c443, 0x897, 0x897, 0x812be0, 0xc2137a18f0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:78 +0x145
github.com/couchbase/goxdcr/simple_utils.IsJSON(0xc3d5e0c443, 0x897, 0x897, 0xc2080bb3a8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:168 +0x6c
github.com/couchbase/goxdcr/parts.getDocMap(0xc3a4aaccc0, 0xc3349b8f00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:987 +0x4fe
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3cc43cfc0, 0x65, 0xc2080bbb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3cc43cfc0, 0xc3d60f0065, 0xc2080bbb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3cc43cfc0, 0xc3d5ca0f50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3cc43cfc0, 0xc3d5ca0f50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3cc43cfc0, 0xc3cc48eaf0, 0xc3cc43d160, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 3356 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc212405860, 0x1dcd6500, 0xc3cc569d20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5de4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5de4b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1289 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c8480, 0x4, 0xc23faaa700, 0xea, 0x100, 0xc290abf800, 0xa2, 0x100, 0xc295081200, 0xc295081140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1231 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c8480, 0x1, 0xc2ba811b00, 0xea, 0x100, 0xc2bdce3800, 0xc7, 0x100, 0xc2be44bd40, 0xc2be44bc80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 325 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xc9, 0xc208177300, 0xec, 0x100, 0xc22fd89000, 0xbd, 0x100, 0xc230afa840, 0xc230afa780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1881 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x0, 0xc288c6cf00, 0xea, 0x100, 0xc2e4488800, 0xa4, 0x100, 0xc2e50cccc0, 0xc2e50ccc00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2466 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36e210b40, 0x0, 0xc378cca000, 0xea, 0x100, 0xc3789b7800, 0x7a, 0x80, 0xc378cb42a0, 0xc378cb41e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1315 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c8480, 0x66, 0xc2ba811d00, 0xec, 0x100, 0xc2c7401800, 0xc8, 0x100, 0xc2672a4720, 0xc2672a4660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1841 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x64, 0xc321de5e00, 0xec, 0x100, 0xc33143c000, 0xc9, 0x100, 0xc331445e00, 0xc331445d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3362 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457cc00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 331 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xcb, 0xc239816100, 0xec, 0x100, 0xc239035000, 0xc9, 0x100, 0xc2397c3b60, 0xc2397c3aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1912 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x6d, 0xc31136b600, 0xec, 0x100, 0xc31c284000, 0xb0, 0x100, 0xc2372e7ce0, 0xc2372e7c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1188 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c8480, 0x3, 0xc2b0faf300, 0xea, 0x100, 0xc2b1c01800, 0x81, 0x100, 0xc2b20854a0, 0xc2b20853e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3128 [runnable]:
reflect.flag.mustBe(0x15, 0x15)
	/usr/local/go/src/reflect/value.go:193
reflect.Value.MapIndex(0x8374c0, 0xc217e63ad0, 0x15, 0x83a640, 0xc212c1ee60, 0x58, 0x0, 0x0, 0x0)
	/usr/local/go/src/reflect/value.go:987 +0x8e
encoding/json.(*mapEncoder).encode(0xc2081040e0, 0xc28a3af8c0, 0x8374c0, 0xc217e63ad0, 0x15, 0x0)
	/usr/local/go/src/encoding/json/encode.go:619 +0x2e5
encoding/json.*mapEncoder.(encoding/json.encode)·fm(0xc28a3af8c0, 0x8374c0, 0xc217e63ad0, 0x15, 0xc217e63a00)
	/usr/local/go/src/encoding/json/encode.go:629 +0x58
encoding/json.(*encodeState).reflectValue(0xc28a3af8c0, 0x8374c0, 0xc217e63ad0, 0x15)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.interfaceEncoder(0xc28a3af8c0, 0x833e80, 0xc212c1ee20, 0x54, 0xc212c1ed00)
	/usr/local/go/src/encoding/json/encode.go:557 +0xd9
encoding/json.(*mapEncoder).encode(0xc2081040e0, 0xc28a3af8c0, 0x8374c0, 0xc217e63530, 0x15, 0x0)
	/usr/local/go/src/encoding/json/encode.go:619 +0x33d
encoding/json.*mapEncoder.(encoding/json.encode)·fm(0xc28a3af8c0, 0x8374c0, 0xc217e63530, 0x15, 0xc217e63500)
	/usr/local/go/src/encoding/json/encode.go:629 +0x58
encoding/json.(*encodeState).reflectValue(0xc28a3af8c0, 0x8374c0, 0xc217e63530, 0x15)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.(*encodeState).marshal(0xc28a3af8c0, 0x8374c0, 0xc217e63530, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:270 +0xca
encoding/json.Marshal(0x8374c0, 0xc217e63530, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:135 +0x92
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3cc43d200, 0xa900c8, 0xc3d5fdbb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:785 +0x34b
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3cc43d200, 0xc3a48b00c8, 0xc3d5fdbb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3cc43d200, 0xc3d0d2b180, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3cc43d200, 0xc3d0d2b180, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3cc43d200, 0xc3cc48ec40, 0xc3cc43d3a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 3353 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2124057a0, 0x1dcd6500, 0xc3cc569da0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5dea50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5dea50)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1269 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c86c0, 0x12e, 0xc2c9274a00, 0xec, 0x100, 0xc2c8a11000, 0xb3, 0x100, 0xc2c928d3e0, 0xc2c928d320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3121 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc3cc2ee300, 0xc3cc6f1a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 2249 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x65, 0xc369492400, 0xec, 0x100, 0xc369908000, 0xc6, 0x100, 0xc369987140, 0xc369987080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 297 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12e, 0xc208177600, 0xec, 0x100, 0xc219ad7000, 0xb3, 0x100, 0xc21a57ff20, 0xc21a57fe60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3295 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21824e100, 0x3b9aca00, 0xc3cc569c50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3cc5c4160, 0x7f4c41c2d8f0, 0xc3cc43cfc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f4c41c2d8f0, 0xc3cc43cfc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 2495 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36e210b40, 0x0, 0xc388409f00, 0xea, 0x100, 0xc387717000, 0xa3, 0x100, 0xc38844df20, 0xc38844de60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 307 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xc8, 0xc228069500, 0xec, 0x100, 0xc211382000, 0xbb, 0x100, 0xc219f6df80, 0xc219f6dec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 108 [select]:
net/http.(*persistConn).writeLoop(0xc20802e580)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 107 [IO wait]:
net.(*pollDesc).Wait(0xc2083993a0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2083993a0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208399340, 0xc2081a5000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc3d4f34c68)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208104220, 0xc2081a5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc208104220, 0xc20802e5d8, 0xc2081a5000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2083b65c0, 0xc2081a5000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20a46f200)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20a46f200, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20802e580)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 991 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0x12c, 0xc213020200, 0xec, 0x100, 0xc293c4b000, 0xc4, 0x100, 0xc294cf7440, 0xc294cf7380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3135 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc3cc2ee600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 115 [IO wait]:
net.(*pollDesc).Wait(0xc208398ae0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208398ae0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208398a80, 0xc20a52d000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc3cc268948)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080381e0, 0xc20a52d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc2080381e0, 0xc208102898, 0xc20a52d000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20a532360, 0xc20a52d000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20a46fda0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20a46fda0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208102840)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 116 [select]:
net/http.(*persistConn).writeLoop(0xc208102840)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3130 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3d4f3b620, 0xc3d4ef7320, 0xc3d4ef7380)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 145 [syscall, 3 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 3196 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066f00, 0x7f4c41c266f0, 0xc3dad8c0a0, 0xc3daa62ea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038138, 0x7f4c41c266f0, 0xc3dad8c0a0, 0xc3daa62ea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f4c41c266f0, 0xc3dad8c0a0, 0xc3daa62ea0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066f60, 0x7f4c41c266f0, 0xc3dad8c0a0, 0xc3daa62ea0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3d66aedc0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 341 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x0, 0xc231a9c400, 0xea, 0x100, 0xc23144f800, 0xb8, 0x100, 0xc231b293e0, 0xc231b29320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2267 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x3, 0xc36cbbba00, 0xea, 0x100, 0xc36d133800, 0xb5, 0x100, 0xc36d558000, 0xc36d475f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1869 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x69, 0xc320d88800, 0xec, 0x100, 0xc328b98000, 0xba, 0x100, 0xc2b1a6ff20, 0xc2b1a6fe60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 836 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x0, 0xc27dad9800, 0xea, 0x100, 0xc27dbd4c00, 0x75, 0x80, 0xc27dc8cfc0, 0xc27dc8cf00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 287 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x65, 0xc213021400, 0xec, 0x100, 0xc22b2fb800, 0xc6, 0x100, 0xc21c9c1ec0, 0xc21c9c1e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2251 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0xcb, 0xc369493c00, 0xec, 0x100, 0xc369cce800, 0xc9, 0x100, 0xc36a07fe60, 0xc36a07fda0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3062 [semacquire]:
sync.(*WaitGroup).Wait(0xc3d5a7da40)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps(0xc3cc680b40, 0xc20800b540, 0x32, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:358 +0xeb9
github.com/couchbase/goxdcr/factory.(*XDCRFactory).SetStartSeqno(0xc208032280, 0x7f4c41c2d9d0, 0xc3cc5c4160, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:575 +0x401
github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7f4c41c2d9d0, 0xc3cc5c4160, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:140 +0x4d
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:179 +0x4b3

goroutine 996 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x66, 0xc210ca9400, 0xec, 0x100, 0xc291f81000, 0xc8, 0x100, 0xc292952ea0, 0xc292952de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3332 [select]:
net/http.(*persistConn).writeLoop(0xc2cf132580)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 279 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x64, 0xc210eed800, 0xec, 0x100, 0xc226996800, 0xc9, 0x100, 0xc227877a40, 0xc227877980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2167 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x64, 0xc270765c00, 0xec, 0x100, 0xc30321b000, 0xc9, 0x100, 0xc2fc64ec60, 0xc2fc64eba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3288 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066f00, 0x7f4c41c266f0, 0xc3db716460, 0xc334d28c30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038138, 0x7f4c41c266f0, 0xc3db716460, 0xc334d28c30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f4c41c266f0, 0xc3db716460, 0xc334d28c30)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066f60, 0x7f4c41c266f0, 0xc3db716460, 0xc334d28c30)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3dad8c000)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 267 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x2, 0xc208176a00, 0xea, 0x100, 0xc21769a000, 0xa7, 0x100, 0xc210f75ce0, 0xc210f75c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3011 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3988058c0, 0x1, 0xc3c429e400, 0xea, 0x100, 0xc3c401d800, 0xc7, 0x100, 0xc3c42a7680, 0xc3c42a75c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1871 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x2, 0xc320d89e00, 0xea, 0x100, 0xc3209fb800, 0xcf, 0x100, 0xc267c493e0, 0xc267c49320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 244 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12f, 0xc213021900, 0xec, 0x100, 0xc213194000, 0xcc, 0x100, 0xc213229ce0, 0xc213229c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 659 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0xc9, 0xc270764900, 0xec, 0x100, 0xc2701f9800, 0xbd, 0x100, 0xc270857c20, 0xc270857b60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 579 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0x12c, 0xc2330e0e00, 0xec, 0x100, 0xc23b1f4000, 0xc4, 0x100, 0xc223c0eea0, 0xc223c0ede0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 676 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x6c, 0xc272952300, 0xec, 0x100, 0xc27270c800, 0xb9, 0x100, 0xc272a4f3e0, 0xc272a4f320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2493 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36e210d80, 0xc9, 0xc383c7f600, 0xec, 0x100, 0xc3864aa000, 0xa4, 0x100, 0xc3864b35c0, 0xc3864b3500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3079 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5de0f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3131 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3cc2ee600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 632 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0x12e, 0xc208054900, 0xec, 0x100, 0xc2631b1800, 0x9c, 0x100, 0xc2637c8c00, 0xc2637c8b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3132 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc3cc2ee600, 0xc3d4ef18f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 3122 [runnable]:
io.ReadFull(0x7f4c41c25b08, 0xc3cf97c058, 0xc334e7e900, 0x41d, 0x41d, 0x41d, 0x0, 0x0)
	/usr/local/go/src/io/io.go:315
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc20839fe78, 0x7f4c41c25b08, 0xc3cf97c058, 0xc3d0daaf40, 0x18, 0x18, 0xc31dc47b1f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:156 +0x76a
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3cc3c7e60, 0xc3d0c67b60)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3364 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457cd80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 251 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12c, 0xc210eed500, 0xec, 0x100, 0xc214b55800, 0xc4, 0x100, 0xc219f7b860, 0xc219f7b7a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3078 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5dea50)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 354 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x1, 0xc23a98a500, 0xea, 0x100, 0xc239dac800, 0xc7, 0x100, 0xc23a9b8fc0, 0xc23a9b8f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 373 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x69, 0xc231a9ca00, 0xec, 0x100, 0xc23ff36800, 0xba, 0x100, 0xc2406b14a0, 0xc2406b13e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 615 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x1, 0xc2330e1100, 0xea, 0x100, 0xc253709800, 0xc7, 0x100, 0xc225d79ec0, 0xc225d79e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 365 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xca, 0xc239816900, 0xec, 0x100, 0xc239863000, 0xbc, 0x100, 0xc24092a000, 0xc240952a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 517 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0x12e, 0xc208177800, 0xeb, 0x100, 0xc237bbd800, 0x23, 0x40, 0xc235de6840, 0xc235de6780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1619 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d440, 0xe, 0xc31136ae00, 0xeb, 0x100, 0xc31149e000, 0x1f4, 0x200, 0xc311649e00, 0xc311649d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2208 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0x12c, 0xc35b608b00, 0xec, 0x100, 0xc35b8b4000, 0xc4, 0x100, 0xc35b9e6780, 0xc35b9e66c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3081 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5de4b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3372 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc221d18f60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3346 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2124055e0, 0x1dcd6500, 0xc3cc569f20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5df4f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5df4f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2221 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x1, 0xc359aa0400, 0xea, 0x100, 0xc358e34800, 0xb2, 0x100, 0xc359b093e0, 0xc359b09320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1571 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d680, 0xd3, 0xc228069100, 0xec, 0x100, 0xc2dc9a8000, 0xba, 0x100, 0xc2dca029c0, 0xc2dca02900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3124 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc3cc2ee300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:493 +0x645
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:257 +0x65a

goroutine 594 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x0, 0xc237a64f00, 0xea, 0x100, 0xc258edc800, 0xa1, 0x100, 0xc21946dec0, 0xc21946de00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 970 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0x12d, 0xc231a9d700, 0xec, 0x100, 0xc27408e000, 0xbc, 0x100, 0xc2740837a0, 0xc2740836e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3120 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3cc2ee300)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1554 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d440, 0x0, 0xc210ca9c00, 0xea, 0x100, 0xc2db73c000, 0xb7, 0x100, 0xc2db7b3140, 0xc2db7b3080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2185 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0x12e, 0xc237a65000, 0xec, 0x100, 0xc340b62800, 0xb3, 0x100, 0xc341455ec0, 0xc341455e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1855 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x65, 0xc321381f00, 0xec, 0x100, 0xc324552800, 0xc6, 0x100, 0xc324cbdf80, 0xc324cbdec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1891 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x68, 0xc321de4900, 0xec, 0x100, 0xc2bb8c4800, 0xc2, 0x100, 0xc32c9f0660, 0xc32c9f05a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3129 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3cc43d200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 940 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x0, 0xc27dad8c00, 0xea, 0x100, 0xc2836f5800, 0xa3, 0x100, 0xc2332965a0, 0xc2332964e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3368 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457d080)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2173 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0x12d, 0xc266581d00, 0xec, 0x100, 0xc2a1a30800, 0xa6, 0x100, 0xc281785440, 0xc281785380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1285 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c86c0, 0x12c, 0xc23faaa400, 0xec, 0x100, 0xc285d50000, 0xb4, 0x100, 0xc285fb7140, 0xc285fb7080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1506 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d680, 0xd8, 0xc2330e0800, 0xec, 0x100, 0xc2b299f200, 0x2a, 0x40, 0xc2b299d680, 0xc2b299d5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3371 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc221d18ea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 934 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x64, 0xc288c6c600, 0xec, 0x100, 0xc21dfc8000, 0xc9, 0x100, 0xc24ae14180, 0xc24ae140c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3294 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21827db80, 0x3b9aca00, 0xc3cc569c50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3cc5c4160, 0x7f4c41c2dc98, 0xc3cc2ee300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f4c41c2dc98, 0xc3cc2ee300)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3168 [select]:
net/http.(*persistConn).roundTrip(0xc3db6926e0, 0xc21181a880, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc334d28ea0, 0xc211660ed0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc334d28ea0, 0x7f4c41c23fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc334d28ea0, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc334d28ea0, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc334d28ea0, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc334d28ea0, 0x0, 0x818220, 0xc3d8a920d8, 0xeb0d80, 0xc20802b590, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3d0cdd357, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8680, 0xd, 0xc2080d8690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b5a0, 0x9ed550, 0xe, 0xc31dd31500, 0xeb0d80, 0x5, 0x66504c, 0x30000c3d8a920d0, 0xc3cc7def00, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b5a0, 0xc3cc453110, 0xc21170b6c0, 0xc3a1fa1e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3cc680b40, 0x2, 0xc3d0ca3a50, 0x64, 0x98, 0xc3d0cdd470, 0x0, 0xc3d4f2f530, 0xc3d5a7da40, 0xc3d59e1740)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 639 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x66, 0xc26d898700, 0xec, 0x100, 0xc26d940000, 0xc8, 0x100, 0xc26da2fe60, 0xc26da2fda0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 596 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0x12d, 0xc213020000, 0xec, 0x100, 0xc2128d0000, 0xcf, 0x100, 0xc22e2985a0, 0xc22e2984e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3119 [runnable]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3d0c00db0, 0xc3d0c66780, 0xc3d0c667e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 1595 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d680, 0xd8, 0xc30f68d600, 0xec, 0x100, 0xc30f882000, 0x1f4, 0x200, 0xc30fa60c00, 0xc30fa60b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2073 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x1, 0xc2a6bec600, 0xe9, 0x100, 0xc303fc1500, 0x1f, 0x20, 0xc304279f80, 0xc304279ec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1604 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d440, 0x9, 0xc30ef6a800, 0xea, 0x100, 0xc30f06c000, 0xc2, 0x100, 0xc30f0b1500, 0xc30f0b1440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3373 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc221d19020)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3331 [runnable]:
net.(*pollDesc).Wait(0xc334e92df0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc334e92df0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc334e92d90, 0xc31ca02000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc212842d78)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3db44a1b8, 0xc31ca02000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc3db44a1b8, 0xc2cf1325d8, 0xc31ca02000, 0x1000, 0x1000, 0x7f4c41c12010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc24b45c3c0, 0xc31ca02000, 0x1000, 0x1000, 0xc1eacbb5e3, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2122b8420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2122b8420, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2cf132580)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1627 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29d75d440, 0xe, 0xc312dba100, 0xeb, 0x100, 0xc312309000, 0x1f4, 0x200, 0xc312d65c20, 0xc312d65b60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 512 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x0, 0xc213021b00, 0xea, 0x100, 0xc2374b1800, 0x2d, 0x40, 0xc235e22ba0, 0xc235e22ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3354 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2124057e0, 0x1dcd6500, 0xc3cc569d00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5de0f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5de0f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 619 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0xc8, 0xc23a98bb00, 0xec, 0x100, 0xc24c0a0800, 0xbb, 0x100, 0xc24aa3ede0, 0xc24aa3ed20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 581 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x65, 0xc2330e1200, 0xec, 0x100, 0xc23b1f5800, 0xc6, 0x100, 0xc22e2e5d40, 0xc22e2e5c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 681 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab38c0, 0xca, 0xc272953f00, 0xec, 0x100, 0xc272cda800, 0xbc, 0x100, 0xc2730bb6e0, 0xc2730bb620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2918 [select]:
net/http.(*persistConn).writeLoop(0xc3a954ebb0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 3370 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc221d18de0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 643 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc23cab3680, 0x64, 0xc223382200, 0xec, 0x100, 0xc2623d2000, 0xc9, 0x100, 0xc2624844e0, 0xc262484420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3351 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc212405720, 0x1dcd6500, 0xc3cc569e80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5dec30, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5dec30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1311 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c86c0, 0xc9, 0xc2c9275800, 0xec, 0x100, 0xc2c8d95800, 0xbd, 0x100, 0xc25cd407e0, 0xc25cd40720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3134 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3cc2ee600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:367 +0x2392
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 3348 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc212405660, 0x1dcd6500, 0xc3cc569d90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5de870, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5de870)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3289 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066f00, 0x7f4c41c266f0, 0xc3db716500, 0xc334d28d00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038138, 0x7f4c41c266f0, 0xc3db716500, 0xc334d28d00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f4c41c266f0, 0xc3db716500, 0xc334d28d00)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066f60, 0x7f4c41c266f0, 0xc3db716500, 0xc334d28d00)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3db716280)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 944 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0xc9, 0xc270764f00, 0xec, 0x100, 0xc281176000, 0xbd, 0x100, 0xc22361d980, 0xc22361d8c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 844 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0x12d, 0xc27e102700, 0xeb, 0x100, 0xc27e121500, 0x1f, 0x20, 0xc27e215140, 0xc27e215080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 985 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x65, 0xc237a65900, 0xec, 0x100, 0xc25cd92000, 0xc6, 0x100, 0xc2711c85a0, 0xc2711c84e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1007 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0xca, 0xc208177c00, 0xec, 0x100, 0xc2a14e2000, 0xbc, 0x100, 0xc2a15b63c0, 0xc2a15b6300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1016 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465b8c0, 0x2, 0xc208055700, 0xea, 0x100, 0xc29f81a800, 0xcf, 0x100, 0xc2a036b740, 0xc2a036b680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3355 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc212405820, 0x1dcd6500, 0xc3cc569d10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5de2d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5de2d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3328 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xeb1a20, 0xc3db5e6540, 0xc3db5e6570, 0xc2080103f0, 0xc3daecc1c0, 0xc20808e7e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:269 +0xa90
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 1046 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0x12f, 0xc2a6bed000, 0xec, 0x100, 0xc2a690f000, 0xce, 0x100, 0xc2a6d62240, 0xc2a6d62180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2967 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3988058c0, 0x66, 0xc3af914500, 0xec, 0x100, 0xc3af407000, 0xc8, 0x100, 0xc3afd069c0, 0xc3afd06900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 893 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc27465bb00, 0xc8, 0xc28430d600, 0xec, 0x100, 0xc28442f800, 0xbb, 0x100, 0xc28512e780, 0xc28512e6c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3290 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208066f00, 0x7f4c41c266f0, 0xc3db7165a0, 0xc334d28dd0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038138, 0x7f4c41c266f0, 0xc3db7165a0, 0xc334d28dd0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7f4c41c266f0, 0xc3db7165a0, 0xc334d28dd0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc208066f60, 0x7f4c41c266f0, 0xc3db7165a0, 0xc334d28dd0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3db716320)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 3166 [sleep]:
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3d0cdd357, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8680, 0xd, 0xc2080d8690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:313 +0x5a0
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b5a0, 0x9ed550, 0xe, 0xc3db556660, 0xeb0d80, 0x5, 0x66504c, 0x30000c3db44a910, 0xc3cc413d80, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b5a0, 0xc3cc453110, 0xc3db57f220, 0xc2080b7e00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3cc680b40, 0x0, 0xc3d0ca38c0, 0x64, 0x160, 0xc3d0cdd470, 0x0, 0xc3d4f2f530, 0xc3d5a7da40, 0xc3d59e1740)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3369 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457d140)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3333 [IO wait]:
net.(*pollDesc).Wait(0xc334e92d80, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc334e92d80, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc334e92d20, 0xc31cf3b000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc21349c118)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3db44a1c8, 0xc31cf3b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc3db44a1c8, 0xc2cf132528, 0xc31cf3b000, 0x1000, 0x1000, 0x7f4c41c12010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc24b45c640, 0xc31cf3b000, 0x1000, 0x1000, 0xc1eacbb5d5, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2122b8540)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2122b8540, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2cf1324d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2241 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0x133, 0xc36be36a00, 0xec, 0x100, 0xc36bea8000, 0xb8, 0x100, 0xc36be9ba40, 0xc36be9b980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1273 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a79c8480, 0x65, 0xc2c99e4700, 0xec, 0x100, 0xc2ca405800, 0xc6, 0x100, 0xc2caca1260, 0xc2caca11a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3352 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc212405760, 0x1dcd6500, 0xc3cc569ea0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5defa0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5defa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3285 [runnable]:
github.com/couchbase/goxdcr/log.(*CommonLogger).Infof(0xc20802b610, 0xa130d0, 0x16, 0xc29d703d58, 0x1, 0x1)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:147
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).stopPipeline(0xeb0fa0, 0xc2080ee630, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:350 +0x90c
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc3d8a21480, 0xc3daa0daa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:583 +0x274
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc3d8a21480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 3334 [select]:
net/http.(*persistConn).writeLoop(0xc2cf1324d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2259 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x0, 0xc36b5cd600, 0xea, 0x100, 0xc36b43f800, 0xb8, 0x100, 0xc36b810180, 0xc36b8100c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2829 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3988058c0, 0x0, 0xc3a2fabd00, 0xea, 0x100, 0xc3a2b06800, 0xb8, 0x100, 0xc3a346e3c0, 0xc3a346e300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3335 [IO wait]:
net.(*pollDesc).Wait(0xc33494c610, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc33494c610, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc33494c5b0, 0xc31d331000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc2137a1b80)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3db44a298, 0xc31d331000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc3db44a298, 0xc210f59b28, 0xc31d331000, 0x1000, 0x1000, 0x7f4c41c12010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc21825a480, 0xc31d331000, 0x1000, 0x1000, 0xc1eacbb5b1, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2145c6c60)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2145c6c60, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc210f59ad0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3118 [runnable]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3cc43cfc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1768 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012240, 0x0, 0xc31c554a00, 0xea, 0x100, 0xc31c581800, 0x49, 0x80, 0xc31c5cf080, 0xc31c5cefc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2196 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6900, 0x2, 0xc210ca9a00, 0xea, 0x100, 0xc34d6a7000, 0xcf, 0x100, 0xc34e164de0, 0xc34e164d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3127 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc3cc43d200, 0xc3cc48ed20, 0xc3cc43d3a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 1934 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc313012480, 0xca, 0xc30ef6b200, 0xec, 0x100, 0xc312c68000, 0xbc, 0x100, 0xc23353bc80, 0xc23353bbc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3001 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc398805b00, 0x12e, 0xc3c448c600, 0xec, 0x100, 0xc3c3ffb000, 0xb3, 0x100, 0xc3c4451c80, 0xc3c4451bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2101 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0x12d, 0xc2a6bed400, 0xec, 0x100, 0xc2fe715000, 0x6e, 0x80, 0xc2fcb41800, 0xc2fcb41740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3358 [chan send]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3cc3c7e60, 0x6870001, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3cc2ee300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3cc2ee300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc219a24d20)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3076 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5dec30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3169 [select]:
net/http.(*persistConn).roundTrip(0xc2cf1324d0, 0xc2117dc870, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc334d29040, 0xc2113c32f0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc334d29040, 0x7f4c41c23fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc334d29040, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc334d29040, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc334d29040, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc334d29040, 0x0, 0x818220, 0xc3db44a050, 0xeb0d80, 0xc20802b590, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3d0cdd357, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8680, 0xd, 0xc2080d8690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b5a0, 0x9ed550, 0xe, 0xc3db78bce0, 0xeb0d80, 0x5, 0x66504c, 0x30000c3db44a048, 0xc3cc7ea800, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b5a0, 0xc3cc453110, 0xc334e269c0, 0xc20815ee00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3cc680b40, 0x3, 0xc3d0ca3b18, 0x29, 0x34, 0xc3d0cdd470, 0x0, 0xc3d4f2f530, 0xc3d5a7da40, 0xc3d59e1740)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 3338 [select]:
net/http.(*persistConn).writeLoop(0xc3db6926e0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2211 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc24e9d6b40, 0xc8, 0xc223383700, 0xec, 0x100, 0xc34ef75800, 0xbb, 0x100, 0xc34faee1e0, 0xc34faee120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3361 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457cb40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2917 [IO wait]:
net.(*pollDesc).Wait(0xc3a96b0fb0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3a96b0fb0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3a96b0f50, 0xc3aa02b000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc3daef9a78)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3a6c8aa80, 0xc3aa02b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc3a6c8aa80, 0xc3a954ec08, 0xc3aa02b000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3aa0989c0, 0xc3aa02b000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3aa02f620)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3aa02f620, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3a954ebb0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3116 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc3cc43cfc0, 0xc3cc48ebd0, 0xc3cc43d160)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 3167 [select]:
net/http.(*persistConn).roundTrip(0xc2cf132580, 0xc2116a8620, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc334d29110, 0xc212cfe060, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc334d29110, 0x7f4c41c23fe8, 0xc20802c090, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc334d29110, 0x27, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc334d29110, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc334d29110, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/goxdcr/utils.doRestCall(0xc334d29110, 0x0, 0x818220, 0xc3d8a920a8, 0xeb0d80, 0xc20802b590, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.InvokeRestWithRetryWithAuth(0xc3d0cdd357, 0x11, 0x9ed550, 0xe, 0x0, 0xc2080d8680, 0xd, 0xc2080d8690, 0xc, 0xeba230, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:299 +0x23d
github.com/couchbase/goxdcr/service_impl.(*CAPIService).send_post(0xc20802b5a0, 0x9ed550, 0xe, 0xc31dd300c0, 0xeb0d80, 0x5, 0x66504c, 0x30000c3d8a920a0, 0xc3cc7d7580, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:330 +0x285
github.com/couchbase/goxdcr/service_impl.(*CAPIService).PreReplicate(0xc20802b5a0, 0xc3cc453110, 0xc21170a4c0, 0xc29d6ffe00, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/capi_service.go:75 +0x282
github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).startSeqnoGetter(0xc3cc680b40, 0x1, 0xc3d0ca3988, 0x64, 0xfc, 0xc3d0cdd470, 0x0, 0xc3d4f2f530, 0xc3d5a7da40, 0xc3d59e1740)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:425 +0x4d6
created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).SetVBTimestamps
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:348 +0xe8d

goroutine 2983 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc398805b00, 0x12c, 0xc3ad72e900, 0xec, 0x100, 0xc3afd8c000, 0x9e, 0x100, 0xc3afd43c80, 0xc3afd43bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3077 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5defa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2555 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36e210d80, 0xca, 0xc38dedc600, 0xec, 0x100, 0xc38f583800, 0xbc, 0x100, 0xc38f793c80, 0xc38f793bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2534 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36e210b40, 0x2, 0xc390f48200, 0xea, 0x100, 0xc391000000, 0xcf, 0x100, 0xc3910f20c0, 0xc3910f2000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3366 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457cf00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3357 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2124058a0, 0x1dcd6500, 0xc3cc569e90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5dee10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5dee10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2459 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36e210d80, 0xc9, 0xc37803b000, 0xec, 0x100, 0xc37892cc00, 0x5c, 0x80, 0xc37894cba0, 0xc37894cae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3350 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc2124056e0, 0x1dcd6500, 0xc3cc569f00, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3cc5df180, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7f4c41c2dd18, 0xc3cc5df180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3080 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5de2d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3360 [runnable]:
github.com/couchbase/goxdcr/log.addTimestampToPrefix(0x9c66e0, 0x7, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:221 +0xcc
github.com/couchbase/goxdcr/log.(*CommonLogger).logMsgf(0xc3cc4413b0, 0x1, 0x9c66e0, 0x7, 0xa62fd0, 0x24, 0xc3a1fa5db8, 0x1, 0x1)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:137 +0x77
github.com/couchbase/goxdcr/log.(*CommonLogger).Infof(0xc3cc4413b0, 0xa62fd0, 0x24, 0xc3a1fa5db8, 0x1, 0x1)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:148 +0x76
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3cc2ee600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:314 +0x70e
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3cc2ee600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457ca80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3133 [runnable]:
encoding/binary.bigEndian.Uint64(0xc334a1ad20, 0x8, 0xec, 0xab)
	/usr/local/go/src/encoding/binary/binary.go:109
github.com/couchbase/gomemcached/client.makeUprEvent(0x57, 0x14b324cf73090000, 0xd603e600d6, 0xc334a1ad20, 0x1f, 0xec, 0xc334a1ad3f, 0x22, 0xcd, 0xc334a1ad61, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:111 +0x158
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3d0cccd80, 0xc3d4ef79e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:476 +0x1cc5
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3367 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457cfc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3365 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc21457ce40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3296 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21824e6e0, 0x3b9aca00, 0xc3cc569c50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3cc5c4160, 0x7f4c41c2dc98, 0xc3cc2ee600, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f4c41c2dc98, 0xc3cc2ee600)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3297 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc21824ede0, 0x3b9aca00, 0xc3cc569c50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3cc5c4160, 0x7f4c41c2d8f0, 0xc3cc43d200, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7f4c41c2d8f0, 0xc3cc43d200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3337 [IO wait]:
net.(*pollDesc).Wait(0xc3db4cda30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3db4cda30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3db4cd9d0, 0xc31d4ef000, 0x1000, 0x1000, 0x0, 0x7f4c41c23df0, 0xc2137a1bc8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3db44a2a0, 0xc31d4ef000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f4c41c25b08, 0xc3db44a2a0, 0xc3db692738, 0xc31d4ef000, 0x1000, 0x1000, 0x7f4c41c12010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc21825a820, 0xc31d4ef000, 0x1000, 0x1000, 0xc1eacbb5b1, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2145c6cc0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2145c6cc0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3db6926e0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3087 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5df180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3086 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5df360)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3085 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5de870)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3084 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5de690)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3083 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5df4f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3082 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3cc5dee10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54
[goport] 2017/04/25 13:41:39 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:41:40.014-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:41:40.015-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:41:40.015-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:41:40.050-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:41:40.054-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:41:40.054-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:41:40.054-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:41:40.054-05:00 [INFO] pollEOF: About to start stdin polling
ReplicationManager 2017-04-25T13:41:40.054-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:41:40.054-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:41:40.055-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:41:40.055-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:41:40.055-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:41:40.055-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:41:40.055-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:41:40.055-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:41:40.055-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:41:40.055-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:41:40.055-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:41:40.055-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:41:40.055-05:00 [INFO] ReplicationManager is running
HttpServer 2017-04-25T13:41:40.056-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:41:40.056-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:41:40.057-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
ReplicationSpecChangeListener 2017-04-25T13:41:40.060-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:41:40.060-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
RemoteClusterChangeListener 2017-04-25T13:41:40.062-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:41:40.062-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
PipelineManager 2017-04-25T13:41:40.396-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:41:55.106-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:41:55.112-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:41:55.210-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:41:55.210-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:41:55.210-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:41:55.259-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:41:55.259-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:41:55.259-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:41:55.259-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:41:55.259-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:41:55.259-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:41:55.260-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:41:55.262-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:41:55.264-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
GenericSupervisor 2017-04-25T13:41:55.266-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:41:55.266-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
GenericSupervisor 2017-04-25T13:41:55.266-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:41:55.266-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:41:55.266-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:41:55.266-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:41:55.266-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:41:55.266-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
CheckpointManager 2017-04-25T13:41:55.266-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
CapiUtils 2017-04-25T13:41:55.301-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:41:55.301-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:41:55.301-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:41:55.301-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
StatisticsManager 2017-04-25T13:41:55.302-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:41:55.302-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
XDCRFactory 2017-04-25T13:41:55.302-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:41:55.302-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291 is constructed, start it
GenericPipeline 2017-04-25T13:41:55.303-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[180 190 221 236 277 306 312 192 197 205 211 242 323 229 235 299 263 278 295 206 210 260 202 337 267 212 304 264 204 233 240 261 307 319 328 259 188 199 253 336 334 201 281 196 223 174 179 302 203 225 230 340 310 172 189 219 255 294 220 291 305 182 191 272 187 227 237 251 245 274 194 239 246 271 286 314 320 273 193 232 266 214 215 265 285 195 290 300 321 250 216 297 276 296 330 248 288 317 181 282 315 322 326 333 171 207 257 178 226 228 311 185 217 252 283 186 234 256 303 332 218 231 313 249 254 316 325 309 224 243 244 213 284 208 176 198 241 209 335 279 280 301 175 177 200 258 222 238 318 269 331 292 308 339 268 270 289 183 184 287 173 293 262 327 247 275 298 324 329 338] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[16 38 100 143 6 81 150 32 60 154 50 66 168 22 33 41 90 144 44 167 35 116 157 14 69 86 91 126 72 137 57 27 40 97 117 58 82 103 114 18 52 134 61 77 124 129 130 138 151 9 105 59 121 122 140 162 25 29 119 15 98 135 139 30 95 109 132 94 49 73 101 107 131 67 88 12 87 149 80 166 51 64 113 161 47 71 145 83 5 46 159 36 43 45 79 89 115 148 164 102 141 4 10 62 65 120 136 155 85 153 163 84 11 54 133 42 75 34 152 13 17 19 20 56 23 28 37 118 2 8 21 53 48 70 96 99 125 158 26 55 63 169 92 127 24 142 106 112 68 128 146 165 170 1 74 110 160 39 78 111 147 0 93 123 31 76 108 156 3 7 104]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[271 286 194 239 246 314 320 273 266 193 232 285 214 215 265 290 300 321 195 250 216 297 276 296 330 248 288 317 181 282 315 322 326 333 171 207 257 178 226 228 311 185 217 252 283 186 234 256 303 332 313 218 231 249 254 316 325 309 224 243 244 213 284 208 198 241 176 335 209 279 280 301 177 200 258 175 222 238 318 269 331 292 308 339 268 270 289 183 184 287 173 293 327 262 298 324 329 247 275 338 190 221 236 180 277 306 312 205 211 242 192 197 323 299 229 235 295 263 278 260 206 210 202 337 267 212 304 264 307 319 328 204 233 240 261 259 336 188 199 253 334 201 281 196 223 174 179 302 340 203 225 230 310 294 172 189 219 255 305 220 291 191 272 182 187 227 237 251 245 274] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[49 73 101 107 131 67 88 12 87 149 80 166 51 64 113 161 47 71 145 83 5 46 159 115 148 164 36 43 45 79 89 102 141 4 155 10 62 65 120 136 85 153 163 84 11 54 133 42 75 34 152 13 17 19 20 56 23 28 37 118 2 8 21 53 158 48 70 96 99 125 26 55 63 169 92 127 24 142 106 112 68 128 146 165 170 1 74 110 160 39 78 111 147 0 93 123 31 76 108 156 3 7 104 16 38 100 143 6 81 150 32 60 154 50 66 168 22 33 41 90 144 44 167 35 116 157 14 69 86 91 126 72 137 57 27 40 97 117 58 82 103 114 18 52 134 138 151 61 77 124 129 130 9 105 59 121 122 140 162 25 29 119 15 98 135 139 30 95 109 132 94]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[active:true doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi filter_expression: optimistic_replication_threshold:256 target_nozzle_per_node:2 checkpoint_interval:1800 worker_batch_size:500]
CheckpointManager 2017-04-25T13:41:55.303-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291...
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:41:55.311-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:41:55.311-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:41:55.311-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:41:55.311-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:41:55.311-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:41:55.311-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:41:55.312-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:41:55.312-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:41:55.312-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:41:55.312-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:41:55.313-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:41:55.313-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:41:55.313-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:41:55.313-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:41:55.317-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:41:55.427-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:41:55.427-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:41:55.427-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:41:55.430-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:41:55.545-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:41:55.553-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:41:55.553-05:00 [INFO] connectionSize=5
TCPConnPoolMgr 2017-04-25T13:41:55.554-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:41:55.554-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:41:55.554-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:41:55.554-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:41:55.556-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:41:55.556-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:41:55.574-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:41:55.631-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:41:55.631-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:41:55.631-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:41:55.631-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:41:55.631-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:41:55.638-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:41:55.657-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:41:55.757-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:41:55.757-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:41:55.757-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:41:55.779-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:41:55.779-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:41:55.951-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:41:55.951-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:41:55.951-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:41:55.980-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:41:55.980-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:41:56.000-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:41:56.000-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291 is started----------
PipelineManager 2017-04-25T13:41:56.000-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:41:56.000-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:41:56.000-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:41:56.451-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:41:56.942-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:41:56.942-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:41:56.942-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:41:56.942-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:41:57 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 110 successful
CapiNozzle 2017-04-25T13:41:57.276-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:41:57.282-05:00 [INFO] Send batch count=114
2017/04/25 13:41:57 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 27 successful
StatisticsManager 2017-04-25T13:41:57.389-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:41:57.399-05:00 [INFO] Send batch count=328
2017/04/25 13:41:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 335 successful
CapiNozzle 2017-04-25T13:41:57.516-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:41:57.529-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:41:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:41:57 UPR_STREAMREQ for vb 269 successful
CapiNozzle 2017-04-25T13:41:57.832-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:41:57.850-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:41:58.032-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:41:58 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 77 successful
CapiNozzle 2017-04-25T13:41:58.074-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:41:58 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 296 successful
CheckpointManager 2017-04-25T13:41:58.226-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
CheckpointManager 2017-04-25T13:41:58.226-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:41:58 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:41:58 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:41:58.338-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:41:58.344-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:41:58.383-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:41:58.398-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
StatisticsManager 2017-04-25T13:41:58.417-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
DcpNozzle 2017-04-25T13:41:58.435-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:41:58.435-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:41:58.731-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:41:58.761-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:41:58.771-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:41:58.771-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:41:58.771-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:41:58.771-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:41:58.771-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:41:58.786-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:41:58.786-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:41:58.790-05:00 [INFO] err_list=[{"time":"2017-04-25T13:41:58.790723215-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:41:58.790-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:41:58.846-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:41:58.846-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:41:58.857-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:41:59.256-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:41:59.256-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:41:59.256-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:41:59.256-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:41:59.256-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:41:59.325-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:41:59.326-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291 {"Errors": "[{\"time\":\"2017-04-25T13:41:58.790723215-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 221, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 16970, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 221, "set_filtered": 0, "set_received_from_dcp": 16970, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:41:59.366-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:41:59.366-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:41:59.386-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:41:59.408-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:41:59.428-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:41:59.767-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:41:59.784-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:42:00.025-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:00.052-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:42:00.244-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:00.245-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:42:00.245-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:42:00.245-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:00.245-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:00.352-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:00.352-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:00.368-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:42:00.962-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:00.973-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:42:01.215-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:01.230-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:42:01.556-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:01.556-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:01.556-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:01.556-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:01.556-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:01.655-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:01.655-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:01.666-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:42:01.933-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:01.961-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:42:02.224-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:42:02.261-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:42:02.430-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:02.430-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:42:02.430-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:42:02.430-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:02.430-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:02.533-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:02.533-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:02.557-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:42:02.702-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:42:02.725-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:42:03.012-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:03.028-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:42:03.260-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:42:03.260-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:42:03.260-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:42:03.260-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:03.260-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:03.360-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:03.360-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:03.380-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:42:03.452-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:03.489-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:42:03.835-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:03.856-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:42:04.053-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:04.053-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:42:04.053-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:42:04.053-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:04.053-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:04.115-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:04.115-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:04.133-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:42:04.246-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:04.264-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:42:04.416-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:04.437-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:42:04.679-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:04.687-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:42:04.687-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:42:04.687-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:04.687-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:04.773-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:04.774-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:04.798-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:42:05.022-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:05.036-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
GenericSupervisor 2017-04-25T13:42:05.314-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:42:05.314-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:42:05.314-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:42:05.320-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:42:05.333-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:42:05.563-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:05.563-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:42:05.563-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:42:05.563-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:05.563-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:05.719-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:05.719-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:05.763-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
ToplogyChangeDetector 2017-04-25T13:42:05.787-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:42:05.789-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:42:05.806-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:42:06.185-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:06.199-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:42:06.368-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:42:06.368-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:42:06.368-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:42:06.368-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:06.368-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:06.457-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:06.457-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:06.488-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:42:06.552-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:06.575-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:42:06.952-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:06.969-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:42:07.221-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:07.221-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:42:07.221-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:42:07.221-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:07.221-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:07.363-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:07.363-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:07.381-05:00 [INFO] Retrying update docs for vb 205 for the 1th time
CapiNozzle 2017-04-25T13:42:07.389-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:07.432-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:42:07.835-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:07.835-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:42:07.835-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:42:07.835-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:07.835-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:07.914-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:07.914-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:07.935-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:42:08.000-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:42:08.013-05:00 [INFO] Retrying update docs for vb 205 for the 2th time
CapiNozzle 2017-04-25T13:42:08.302-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:08.313-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:42:08.483-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:42:08.494-05:00 [INFO] Retrying update docs for vb 205 for the 3th time
CapiNozzle 2017-04-25T13:42:08.742-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:08.753-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
PipelineManager 2017-04-25T13:42:08.791-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:42:08.791-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:42:08.791-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:08.791-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
PipelineRuntimeCtx 2017-04-25T13:42:08.791-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:42:08.791-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:08.791-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:42:08.791-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:08.791302345 -0500 CDT
GenericSupervisor 2017-04-25T13:42:08.791-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:08.791-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:42:08.791-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:08.791-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:42:08.791-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:08.791-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:42:08.791-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:42:08.791-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:42:08.791-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:42:08.791-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-259992291 has been stopped
GenericPipeline 2017-04-25T13:42:08.791-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-259992291 is stopped
PipelineManager 2017-04-25T13:42:08.791-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:42:08.791-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:41:58.790723215-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:42:08.792-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:08.792-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
GenericPipeline 2017-04-25T13:42:08.792-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
GenericPipeline 2017-04-25T13:42:08.792-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
GenericPipeline 2017-04-25T13:42:08.792-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-259992291
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:08.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:42:08.792-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:08.792-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:42:08.793-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:08.793-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:42:08.793-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:08.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:42:08.793-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:08.793-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:42:08.793-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:08.793-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:08.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:42:08.793-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:08.793319971 -0500 CDT
DcpNozzle 2017-04-25T13:42:08.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:42:08.794-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:08.793644379 -0500 CDT
CapiNozzle 2017-04-25T13:42:08.794-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:08.794-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:42:08.794-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:08.794-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:08.794-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:08.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:42:08.795-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:08.795-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:42:08.798-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:42:08.798-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:42:08.798-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:08.798-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:42:08.799-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:08.799-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:08.799893027 -0500 CDT
DcpNozzle 2017-04-25T13:42:08.799-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:42:08.799-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:08.800-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:42:08 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:42:08 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:42:08.820-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:08.820-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:08.820584213 -0500 CDT
DcpNozzle 2017-04-25T13:42:08.820-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:42:08.820-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:08.820-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
XDCRFactory 2017-04-25T13:42:08.908-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:08.909-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:42:08.909-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:42:08.956-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:08.957-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:42:08.957-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:08.957-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:42:08.957-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:08.957-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:42:08.957-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:42:08.958-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:42:08.958-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
GenericSupervisor 2017-04-25T13:42:08.960-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:42:08.960-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
GenericSupervisor 2017-04-25T13:42:08.960-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:08.960-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:08.960-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:08.960-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:08.960-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:42:08.960-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
CheckpointManager 2017-04-25T13:42:08.960-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
CapiUtils 2017-04-25T13:42:08.990-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:08.991-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:08.991-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:42:08.991-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
StatisticsManager 2017-04-25T13:42:08.992-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:42:08.992-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
XDCRFactory 2017-04-25T13:42:08.992-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:42:08.992-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722 is constructed, start it
GenericPipeline 2017-04-25T13:42:08.993-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[98 36 97 100 101 107 157 10 66 33 123 48 73 0 59 74 77 102 108 37 70 86 150 28 31 44 50 115 12 23 132 49 88 145 169 21 76 85 109 2 45 124 40 51 94 133 141 126 4 5 20 35 65 122 95 110 148 130 136 6 29 42 43 63 87 83 22 30 138 24 62 131 153 161 3 106 118 34 72 140 9 38 11 61 46 79 139 41 104 119 15 17 54 75 163 80 99 18 160 162 27 93 96 58 117 92 113 146 120 137 47 71 39 105 13 78 90 142 57 151 170 82 164 165 19 89 7 14 103 81 1 25 128 32 114 16 68 127 26 158 8 69 84 55 116 155 91 143 144 149 52 111 134 135 152 154 156 125 129 60 67 112 147 159 64 121 166 168 53 56 167] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[249 272 276 246 256 242 328 208 218 277 338 285 232 264 191 273 262 330 213 245 286 332 200 267 219 220 225 240 182 303 260 193 226 228 300 305 209 199 202 306 266 292 257 181 196 254 315 325 214 179 203 247 265 280 310 204 237 339 180 185 217 282 238 284 326 290 313 173 194 283 184 230 248 287 308 275 295 321 171 190 227 259 302 309 333 197 293 331 187 216 304 334 252 329 175 206 222 307 177 258 314 215 251 318 340 210 223 289 174 241 319 205 188 212 288 255 294 234 235 236 316 324 172 176 291 311 317 183 207 296 337 231 281 243 195 279 298 327 312 336 299 233 244 271 274 268 269 270 322 263 320 297 192 229 253 261 335 211 189 201 178 186 221 224 323 278 301 198 239 250]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[185 180 217 282 238 284 326 173 194 283 290 313 184 230 248 287 308 321 275 295 171 190 227 259 302 309 197 293 331 333 187 216 304 334 252 329 175 206 222 307 177 215 251 258 314 318 210 223 289 340 241 319 174 205 188 212 288 234 235 236 255 294 316 176 291 311 317 324 172 207 296 337 183 231 281 243 195 279 298 327 312 336 233 244 299 268 269 270 271 274 263 320 322 192 229 253 261 297 211 335 178 186 189 201 323 221 224 198 239 250 278 301 272 276 249 242 246 256 328 218 277 338 208 285 232 264 191 273 262 330 286 332 213 245 267 200 225 240 219 220 182 303 193 226 228 260 300 305 209 199 202 266 292 306 181 196 254 257 214 315 325 247 265 280 310 179 203 339 204 237] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[83 138 22 30 131 24 62 153 161 118 3 106 140 34 72 9 38 11 61 139 46 79 119 41 104 54 75 163 15 17 80 99 162 18 160 96 27 93 58 117 146 92 113 120 137 47 71 39 105 90 142 13 78 170 57 151 82 164 165 19 89 103 7 14 81 1 25 128 32 114 127 16 68 26 158 84 8 69 155 55 116 144 149 91 143 134 52 111 154 156 135 152 125 129 112 147 159 60 67 166 168 64 121 167 53 56 98 100 101 107 36 97 157 10 66 33 123 48 73 74 77 102 108 0 59 86 150 37 70 28 50 115 31 44 132 12 23 49 88 145 169 85 109 21 76 124 2 45 94 133 141 40 51 20 35 65 122 126 4 5 148 95 110 42 43 63 87 130 136 6 29]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi checkpoint_interval:1800 optimistic_replication_threshold:256 stats_interval:1000 target_nozzle_per_node:2 log_level:Info filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:42:08.993-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722...
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:09.007-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:42:09.007-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:09.007-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:42:09.007-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:42:09.007-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:42:09.007-05:00 [INFO] Checkpointing starts in 1802.576 sec
PipelineRuntimeCtx 2017-04-25T13:42:09.007-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:42:09.007-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:42:09.008-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:42:09.008-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:42:09.009-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:42:09.010-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:42:09.010-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:42:09.010-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:42:09.010-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
StatisticsManager 2017-04-25T13:42:09.411-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:42:09.414-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:42:09.416-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:09.416-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:09.416-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:42:09.430-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:42:09.481-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:42:09.481-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:42:09.481-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:42:09.512-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:09.512-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:42:09.536-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:42:09.553-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:42:09.553-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:09.553-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:42:09.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:42:09.554-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:42:09.554-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:42:09.584-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:42:09.584-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:42:09.584-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:42:09.603-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:42:09.603-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:42:09.685-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:42:09.685-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:42:09.685-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:42:09.685-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:42:09 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:42:09.748-05:00 [INFO] Send batch count=100
DcpNozzle 2017-04-25T13:42:09.755-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:42:09.755-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:09.755-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:42:09.756-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:42:09.756-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:42:09.756-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:42:09.756-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722 is started----------
PipelineManager 2017-04-25T13:42:09.756-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:42:09.756-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:42:09.756-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CapiNozzle 2017-04-25T13:42:09.772-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:09.772-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:09.790-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:42:09 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:42:09 UPR_STREAMREQ for vb 216 successful
CapiNozzle 2017-04-25T13:42:09.993-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:10.024-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:42:10.041-05:00 [INFO] Send batch count=87
2017/04/25 13:42:10 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 321 successful
StatisticsManager 2017-04-25T13:42:10.148-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:42:10 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 324 successful
PipelineManager 2017-04-25T13:42:10.211-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:42:10 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 45 successful
CapiNozzle 2017-04-25T13:42:10.376-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:42:10 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 340 successful
CapiNozzle 2017-04-25T13:42:10.402-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:42:10.437-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:10.465-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:42:10 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 61 successful
CapiNozzle 2017-04-25T13:42:10.655-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:10.656-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:42:10.656-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:42:10.656-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:42:10.656-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
2017/04/25 13:42:10 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 74 successful
PipelineManager 2017-04-25T13:42:10.671-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:42:10.671-05:00 [INFO] Send batch count=500
2017/04/25 13:42:10 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 176 successful
PipelineManager 2017-04-25T13:42:10.676-05:00 [INFO] err_list=[{"time":"2017-04-25T13:42:10.676142051-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:42:10.677-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:42:10.734-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:10.748-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:10.748-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:10.767-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:42:10.774-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:42:10 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 290 successful
CheckpointManager 2017-04-25T13:42:10.923-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
CheckpointManager 2017-04-25T13:42:10.923-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:42:10 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:42:10 UPR_STREAMREQ for vb 198 successful
StatisticsManager 2017-04-25T13:42:11.017-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:42:11.018-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722 {"Errors": "[{\"time\":\"2017-04-25T13:42:10.676142051-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 12, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 42, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 3715, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 42, "set_filtered": 0, "set_received_from_dcp": 3715, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:42:11.018-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:11.035-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
DcpNozzle 2017-04-25T13:42:11.065-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:42:11.065-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:42:11.496-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:11.512-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:42:11.543-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:11.543-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:42:11.543-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:42:11.543-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:11.543-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:11.618-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:11.618-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:11.636-05:00 [INFO] Retrying update docs for vb 207 for the 1th time
CapiNozzle 2017-04-25T13:42:12.033-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:12.053-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:42:12.117-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:42:12.138-05:00 [INFO] Retrying update docs for vb 207 for the 2th time
CapiNozzle 2017-04-25T13:42:12.881-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:12.881-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:12.881-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:12.881-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:12.881-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:12.957-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:12.957-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:12.981-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:42:13.019-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:42:13.036-05:00 [INFO] Retrying update docs for vb 207 for the 3th time
CapiNozzle 2017-04-25T13:42:13.726-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:13.741-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:42:14.025-05:00 [ERROR] batchUpdateDocs for vb 207 failed with err receieved unexpected status code, 500, from update docs request for vb 207
.
CapiNozzle 2017-04-25T13:42:14.025-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 207 after 3 retries
CapiNozzle 2017-04-25T13:42:14.025-05:00 [ERROR] Raise error condition batch update docs failed for vb 207 after 3 retries
GenericSupervisor 2017-04-25T13:42:14.025-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:14.025-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:14.550-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:14.587-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:14.587-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:14.602-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:42:14.628-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:42:14.954-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:14.954-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:42:14.954-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:42:14.954-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:14.954-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:15.012-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:15.012-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:15.031-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:42:15.113-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:15.130-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:42:15.587-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:15.635-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:42:15.769-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:15.796-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:42:16.185-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:16.200-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:42:16.377-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:16.377-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:42:16.377-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:42:16.377-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:16.377-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:16.451-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:16.451-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:16.464-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
CapiNozzle 2017-04-25T13:42:16.559-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:16.560-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:42:16.560-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:42:16.560-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:16.560-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:16.641-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:16.642-05:00 [ERROR] batchUpdateDocs for vb 12 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:16.663-05:00 [INFO] Retrying update docs for vb 12 for the 1th time
CapiNozzle 2017-04-25T13:42:16.806-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:42:16.826-05:00 [INFO] Retrying update docs for vb 208 for the 2th time
CapiNozzle 2017-04-25T13:42:17.142-05:00 [ERROR] batchUpdateDocs for vb 12 failed with err receieved unexpected status code, 500, from update docs request for vb 12
.
CapiNozzle 2017-04-25T13:42:17.153-05:00 [INFO] Retrying update docs for vb 12 for the 2th time
CapiNozzle 2017-04-25T13:42:17.584-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:42:17.604-05:00 [INFO] Retrying update docs for vb 208 for the 3th time
CapiNozzle 2017-04-25T13:42:17.794-05:00 [ERROR] batchUpdateDocs for vb 12 failed with err receieved unexpected status code, 500, from update docs request for vb 12
.
CapiNozzle 2017-04-25T13:42:17.817-05:00 [INFO] Retrying update docs for vb 12 for the 3th time
CapiNozzle 2017-04-25T13:42:18.051-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:42:18.052-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 208 after 3 retries
CapiNozzle 2017-04-25T13:42:18.052-05:00 [ERROR] Raise error condition batch update docs failed for vb 208 after 3 retries
GenericSupervisor 2017-04-25T13:42:18.052-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:18.052-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:18.149-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:18.150-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:18.160-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:42:18.280-05:00 [ERROR] batchUpdateDocs for vb 12 failed with err receieved unexpected status code, 500, from update docs request for vb 12
.
CapiNozzle 2017-04-25T13:42:18.280-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 12 after 3 retries
CapiNozzle 2017-04-25T13:42:18.280-05:00 [ERROR] Raise error condition batch update docs failed for vb 12 after 3 retries
GenericSupervisor 2017-04-25T13:42:18.280-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:18.280-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:18.353-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:18.353-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:18.364-05:00 [INFO] Retrying update docs for vb 10 for the 1th time
CapiNozzle 2017-04-25T13:42:18.489-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:42:18.500-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:42:18.657-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err receieved unexpected status code, 500, from update docs request for vb 10
.
CapiNozzle 2017-04-25T13:42:18.668-05:00 [INFO] Retrying update docs for vb 10 for the 2th time
CapiNozzle 2017-04-25T13:42:18.836-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:42:18.847-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:42:18.992-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err receieved unexpected status code, 500, from update docs request for vb 10
.
CapiNozzle 2017-04-25T13:42:19.003-05:00 [INFO] Retrying update docs for vb 10 for the 3th time
GenericSupervisor 2017-04-25T13:42:19.008-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:42:19.008-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:42:19.008-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:42:19.191-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:42:19.191-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:42:19.191-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:42:19.191-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:19.191-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:19.274-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:19.275-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:19.285-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:42:19.352-05:00 [ERROR] batchUpdateDocs for vb 10 failed with err receieved unexpected status code, 500, from update docs request for vb 10
.
CapiNozzle 2017-04-25T13:42:19.352-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 10 after 3 retries
CapiNozzle 2017-04-25T13:42:19.352-05:00 [ERROR] Raise error condition batch update docs failed for vb 10 after 3 retries
GenericSupervisor 2017-04-25T13:42:19.352-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:19.352-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:19.390-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 112
CapiNozzle 2017-04-25T13:42:19.474-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:19.474-05:00 [ERROR] batchUpdateDocs for vb 112 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:19.485-05:00 [INFO] Retrying update docs for vb 112 for the 1th time
ToplogyChangeDetector 2017-04-25T13:42:19.553-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:42:19.602-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:42:19.613-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:42:20.020-05:00 [ERROR] batchUpdateDocs for vb 112 failed with err receieved unexpected status code, 500, from update docs request for vb 112
.
CapiNozzle 2017-04-25T13:42:20.031-05:00 [INFO] Retrying update docs for vb 112 for the 2th time
CapiNozzle 2017-04-25T13:42:20.218-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:42:20.228-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:42:20.636-05:00 [ERROR] batchUpdateDocs for vb 112 failed with err receieved unexpected status code, 500, from update docs request for vb 112
.
CapiNozzle 2017-04-25T13:42:20.647-05:00 [INFO] Retrying update docs for vb 112 for the 3th time
PipelineManager 2017-04-25T13:42:20.677-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:42:20.677-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:42:20.677-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:20.677-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
PipelineRuntimeCtx 2017-04-25T13:42:20.677-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:42:20.677-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:42:20.677-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:20.677-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:42:20.677-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:20.677777471 -0500 CDT
GenericSupervisor 2017-04-25T13:42:20.677-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:20.677-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:42:20.677-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:20.678-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:42:20.678-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:20.678-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:42:20.678-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:42:20.678-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:42:20.678-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-957869722 has been stopped
GenericPipeline 2017-04-25T13:42:20.678-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-957869722 is stopped
PipelineManager 2017-04-25T13:42:20.678-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:42:20.678-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:10.676142051-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:42:20.678-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:20.678-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
GenericPipeline 2017-04-25T13:42:20.678-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
GenericPipeline 2017-04-25T13:42:20.678-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
GenericPipeline 2017-04-25T13:42:20.679-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-957869722
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:20.679-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:42:20.679-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:20.679-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:42:20.679-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:20.680-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:42:20.680-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:20.680-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:20.680-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:20.680130219 -0500 CDT
DcpNozzle 2017-04-25T13:42:20.680-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:20.680460131 -0500 CDT
CapiNozzle 2017-04-25T13:42:20.680-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:20.681-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:42:20.681-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:20.681-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:42:20.681-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:20.681-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.681-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:20.682-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:42:20.682-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:20.682-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:42:20.686-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:20.686-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:20.686-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:20.686055606 -0500 CDT
DcpNozzle 2017-04-25T13:42:20.686-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:20.686091659 -0500 CDT
DcpNozzle 2017-04-25T13:42:20.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:42:20.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:42:20.686-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:20.686-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:42:20.686-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:20.686-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:42:20 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:42:20 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:42:20 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:42:20 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:42:20 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:42:20 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:42:20 Stream Closed for vb 175 StreamEnd simulated
XDCRFactory 2017-04-25T13:42:20.825-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:20.825-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:42:20.825-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
CapiNozzle 2017-04-25T13:42:20.835-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:42:20.835-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:42:20.835-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:20.835-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:42:20.868-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:20.868-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:42:20.868-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:20.868-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:42:20.868-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:20.868-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:42:20.868-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:42:20.868-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:42:20.869-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
GenericSupervisor 2017-04-25T13:42:20.870-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:42:20.870-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
GenericSupervisor 2017-04-25T13:42:20.870-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:20.870-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:20.870-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:20.870-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:20.870-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:42:20.870-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
CheckpointManager 2017-04-25T13:42:20.870-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
CapiUtils 2017-04-25T13:42:20.892-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:20.892-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:20.892-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:42:20.892-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
StatisticsManager 2017-04-25T13:42:20.893-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:42:20.893-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
XDCRFactory 2017-04-25T13:42:20.893-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:42:20.893-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967 is constructed, start it
GenericPipeline 2017-04-25T13:42:20.894-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[277 310 225 320 324 246 231 217 243 244 280 281 294 207 272 291 293 297 334 309 321 194 249 319 221 256 179 284 175 264 273 286 311 339 219 248 338 209 283 307 327 251 300 329 201 305 178 266 296 306 180 232 323 325 331 191 211 241 282 287 299 185 228 276 303 226 237 252 274 235 290 335 292 313 337 314 188 214 259 318 261 196 218 239 333 177 171 203 312 200 206 265 279 190 216 271 234 255 229 308 289 340 202 247 213 195 258 193 204 220 253 254 182 275 288 315 227 317 187 208 330 250 257 268 189 245 278 173 181 215 224 326 183 222 316 298 172 210 197 198 270 233 236 262 186 322 332 285 212 328 176 199 205 238 269 174 192 240 242 302 301 260 304 223 295 184 230 263 267 336] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[92 91 99 106 133 135 38 107 7 88 95 21 49 100 152 6 76 112 139 66 40 56 159 12 36 151 5 94 16 85 28 53 61 23 67 119 54 0 98 26 164 69 78 121 63 74 73 142 153 102 52 82 126 143 145 122 109 113 128 163 104 110 124 55 79 147 148 161 3 41 155 37 33 156 2 125 15 48 43 68 81 114 120 132 150 9 157 166 65 22 62 71 72 86 130 20 131 169 154 30 111 87 96 34 24 51 170 75 93 129 167 39 70 123 42 44 58 60 137 1 27 97 160 25 146 165 108 64 57 31 90 127 149 19 83 115 158 77 47 84 116 141 17 144 162 168 138 50 59 140 11 18 134 32 29 89 103 105 8 45 46 101 136 14 13 80 117 4 118 10 35]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[256 221 284 179 175 264 273 286 311 339 338 209 219 248 283 307 327 329 251 300 201 305 296 306 178 266 323 180 232 325 331 211 241 282 287 299 191 185 228 276 303 252 274 226 237 335 235 290 292 313 337 314 188 214 259 261 318 239 333 177 196 218 171 203 312 200 206 265 279 216 271 190 255 234 229 308 289 340 202 213 247 195 258 220 253 254 193 204 275 288 315 182 227 317 187 208 330 250 268 257 189 245 278 173 181 215 224 326 183 222 316 298 210 172 197 198 270 233 236 262 186 322 332 285 212 328 205 238 269 176 199 174 192 240 242 301 302 260 304 295 223 263 267 336 184 230 277 310 225 320 324 231 246 243 244 280 281 294 217 207 272 291 293 297 334 309 321 194 249 319] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[16 94 28 85 23 53 61 54 67 119 0 98 26 164 121 63 69 78 73 74 102 142 153 52 82 145 122 126 143 128 163 104 109 113 55 110 124 148 161 3 79 147 37 41 155 2 33 156 15 125 43 48 114 120 132 150 9 68 81 65 157 166 71 72 86 130 20 22 62 131 169 30 154 111 34 87 96 24 51 75 170 167 39 93 129 42 70 123 60 137 1 44 58 160 25 27 97 108 146 165 64 31 57 149 19 90 127 158 77 83 115 116 141 17 47 84 168 138 144 162 140 11 50 59 18 32 134 103 105 8 29 89 45 136 14 46 101 117 4 13 80 10 118 35 92 106 133 135 38 91 99 7 107 21 88 95 152 6 49 100 139 66 76 112 159 12 40 56 5 36 151]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[source_nozzle_per_node:2 target_nozzle_per_node:2 replication_type:capi filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:42:20.894-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967...
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:20.902-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
CheckpointManager 2017-04-25T13:42:20.902-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:42:20.902-05:00 [INFO] Checkpointing starts in 1804.572 sec
PipelineRuntimeCtx 2017-04-25T13:42:20.902-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:42:20.902-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:42:20.903-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:42:20.903-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:42:20.903-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:42:20.903-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:42:20.903-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:20.903-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:42:20.903-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:42:20.903-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:42:20.904-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:42:20.906-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:42:20.921-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
ToplogyChangeDetector 2017-04-25T13:42:20.925-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:42:20.926-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:20.926-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:20.926-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:42:20.932-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:42:20.933-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:42:20.933-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:42:20.956-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:42:20.957-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:20.957-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:21.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:42:21.043-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:21.043-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:42:21.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:42:21.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CheckpointManager 2017-04-25T13:42:21.047-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:42:21.047-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:42:21.064-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:42:21.064-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:42:21.064-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:42:21.082-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:42:21.082-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:42:21.214-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:42:21.215-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:42:21.215-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:42:21.215-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:42:21.234-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:42:21.234-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:21.234-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:42:21.258-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:42:21.258-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:42:21.258-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:42:21.258-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967 is started----------
PipelineManager 2017-04-25T13:42:21.258-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:42:21.258-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:42:21.258-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:42:21 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:42:21.290-05:00 [INFO] Send batch count=102
CapiNozzle 2017-04-25T13:42:21.341-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:21.343-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:21.369-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:42:21 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 108 successful
CapiNozzle 2017-04-25T13:42:21.466-05:00 [INFO] Send batch count=3
2017/04/25 13:42:21 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 213 successful
CapiNozzle 2017-04-25T13:42:21.536-05:00 [INFO] Send batch count=500
2017/04/25 13:42:21 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 23 successful
CapiNozzle 2017-04-25T13:42:21.596-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:42:21 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 226 successful
CapiNozzle 2017-04-25T13:42:21.639-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:42:21 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 249 successful
CapiNozzle 2017-04-25T13:42:21.894-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:21.942-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:42:21 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:42:21 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 256 successful
StatisticsManager 2017-04-25T13:42:22.046-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:42:22 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 264 successful
CapiNozzle 2017-04-25T13:42:22.236-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:42:22.270-05:00 [INFO] Retrying update docs for vb 309 for the 1th time
2017/04/25 13:42:22 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 276 successful
CapiNozzle 2017-04-25T13:42:22.415-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:22.415-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:22.415-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:22.415-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:42:22.415-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:42:22 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 88 successful
PipelineManager 2017-04-25T13:42:22.466-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:42:22.466-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:42:22.467-05:00 [INFO] err_list=[{"time":"2017-04-25T13:42:22.467867787-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:42:22.468-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:42:22 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:42:22.643-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:22.644-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:42:22 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 93 successful
CapiNozzle 2017-04-25T13:42:22.670-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:42:22 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 297 successful
CheckpointManager 2017-04-25T13:42:22.790-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
CheckpointManager 2017-04-25T13:42:22.790-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:42:22.790-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:42:22.815-05:00 [INFO] Retrying update docs for vb 309 for the 2th time
2017/04/25 13:42:22 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:42:22 UPR_STREAMREQ for vb 99 successful
StatisticsManager 2017-04-25T13:42:22.908-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:42:22.908-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967 {"Errors": "[{\"time\":\"2017-04-25T13:42:22.467867787-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 251, "dcp_datach_length": 0, "dcp_dispatch_time": 23, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 354, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 4, "docs_received_from_dcp": 22795, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 354, "set_filtered": 0, "set_received_from_dcp": 22795, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

DcpNozzle 2017-04-25T13:42:22.999-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:42:22.999-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:42:23.169-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:23.194-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:42:23.303-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:42:23.322-05:00 [INFO] Retrying update docs for vb 309 for the 3th time
CapiNozzle 2017-04-25T13:42:23.697-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:23.730-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:42:23.903-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:42:23.903-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 309 after 3 retries
CapiNozzle 2017-04-25T13:42:23.903-05:00 [ERROR] Raise error condition batch update docs failed for vb 309 after 3 retries
GenericSupervisor 2017-04-25T13:42:23.903-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:23.903-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:24.023-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:24.023-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:24.039-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:42:24.167-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:24.168-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:42:24.168-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:42:24.168-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:24.168-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:24.243-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:24.243-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:24.261-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:42:25.114-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:25.146-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
PipelineManager 2017-04-25T13:42:25.259-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:22.467867787-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:42:25.446-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:25.465-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:42:25.679-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:25.698-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:42:25.863-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:25.880-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:42:26.083-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:26.083-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:42:26.083-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:42:26.083-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:26.083-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:26.133-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:26.144-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:26.144-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:26.144-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:26.144-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:26.158-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:26.158-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:26.192-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:42:26.209-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:26.209-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:26.240-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:42:26.575-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:26.588-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:42:26.860-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:26.877-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:42:27.013-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:27.038-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:42:27.167-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:27.185-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:42:27.357-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:27.357-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:42:27.357-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:42:27.357-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:27.358-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:27.442-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:27.442-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:27.460-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:42:27.612-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:27.612-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:42:27.612-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:42:27.612-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:27.612-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:27.721-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:27.721-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:27.736-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:42:27.993-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:28.004-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:42:28.336-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:42:28.349-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:42:28.691-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:28.706-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:42:28.861-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:42:28.879-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:42:29.204-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:29.204-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:42:29.204-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:42:29.204-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:29.204-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:29.269-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:29.270-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:29.289-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:42:29.291-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:42:29.291-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:42:29.291-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:42:29.291-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:29.291-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:29.355-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:29.355-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:29.371-05:00 [INFO] Retrying update docs for vb 7 for the 1th time
CapiNozzle 2017-04-25T13:42:29.616-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:42:29.647-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:42:29.804-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err receieved unexpected status code, 500, from update docs request for vb 7
.
CapiNozzle 2017-04-25T13:42:29.821-05:00 [INFO] Retrying update docs for vb 7 for the 2th time
CapiNozzle 2017-04-25T13:42:29.969-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:42:29.979-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:42:30.281-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err receieved unexpected status code, 500, from update docs request for vb 7
.
CapiNozzle 2017-04-25T13:42:30.292-05:00 [INFO] Retrying update docs for vb 7 for the 3th time
CapiNozzle 2017-04-25T13:42:30.473-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:42:30.473-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:42:30.473-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:42:30.473-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:30.473-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:30.559-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:30.560-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:30.570-05:00 [INFO] Retrying update docs for vb 308 for the 1th time
CapiNozzle 2017-04-25T13:42:30.639-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err receieved unexpected status code, 500, from update docs request for vb 7
.
CapiNozzle 2017-04-25T13:42:30.639-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 7 after 3 retries
CapiNozzle 2017-04-25T13:42:30.639-05:00 [ERROR] Raise error condition batch update docs failed for vb 7 after 3 retries
GenericSupervisor 2017-04-25T13:42:30.639-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:30.639-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:30.713-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:30.714-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:30.724-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:42:30.850-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:42:30.860-05:00 [INFO] Retrying update docs for vb 308 for the 2th time
GenericSupervisor 2017-04-25T13:42:30.904-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:42:30.904-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:42:30.905-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:42:31.030-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:42:31.460-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:31.471-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:42:31.672-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:42:31.683-05:00 [INFO] Retrying update docs for vb 308 for the 3th time
CapiNozzle 2017-04-25T13:42:31.832-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:31.843-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:42:32.005-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:42:32.005-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 308 after 3 retries
CapiNozzle 2017-04-25T13:42:32.005-05:00 [ERROR] Raise error condition batch update docs failed for vb 308 after 3 retries
GenericSupervisor 2017-04-25T13:42:32.005-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:32.005-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:32.147-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:32.147-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:32.158-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
CapiNozzle 2017-04-25T13:42:32.246-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:32.247-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:42:32.247-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:42:32.247-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:32.247-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:32.327-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:32.328-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:32.338-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
PipelineManager 2017-04-25T13:42:32.468-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:42:32.468-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:42:32.468-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:32.468-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
PipelineRuntimeCtx 2017-04-25T13:42:32.468-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:42:32.468-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:32.468-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:42:32.468-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:32.46845777 -0500 CDT
GenericSupervisor 2017-04-25T13:42:32.468-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:32.468-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:42:32.468-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:32.468-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:42:32.468-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:32.468-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:42:32.468-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:42:32.468-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:42:32.468-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:42:32.468-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-868739967 has been stopped
GenericPipeline 2017-04-25T13:42:32.468-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-868739967 is stopped
PipelineManager 2017-04-25T13:42:32.469-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:42:32.469-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:22.467867787-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:42:32.469-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:32.469-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
GenericPipeline 2017-04-25T13:42:32.469-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:42:32.469-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:42:32.469-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-868739967
GenericPipeline 2017-04-25T13:42:32.469-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:42:32.469-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:42:32.469-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:42:32.469-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:32.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:32.470-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:42:32.470-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:32.470-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:42:32.470-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.470-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:32.470-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:32.470275944 -0500 CDT
DcpNozzle 2017-04-25T13:42:32.470-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:42:32.471-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:42:32.471-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:42:32.471-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.471-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:42:32.471-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:32.472-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:42:32.475-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:32.475-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:32.475261334 -0500 CDT
DcpNozzle 2017-04-25T13:42:32.475-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:42:32.475-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:32.475-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:42:32 Stream Closed for vb 0 StreamEnd simulated
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:32.475-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:42:32.475-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:32.46984101 -0500 CDT
CapiNozzle 2017-04-25T13:42:32.476-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:42:32.476-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:32.476-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:42:32.476-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:32.476-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:42:32.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:42:32.485-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:42:32.485-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:42:32.488-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:32.488-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:32.488406881 -0500 CDT
DcpNozzle 2017-04-25T13:42:32.488-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:42:32.488-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:32.488-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:42:32 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:42:32.591-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:32.592-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:42:32.592-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:42:32.670-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:32.671-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:42:32.671-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:32.671-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:42:32.671-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:32.671-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:42:32.671-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:42:32.671-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:42:32.672-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
GenericSupervisor 2017-04-25T13:42:32.674-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:42:32.674-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
GenericSupervisor 2017-04-25T13:42:32.674-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:32.674-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:32.674-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:32.674-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:32.674-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:42:32.674-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
CheckpointManager 2017-04-25T13:42:32.674-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
CapiUtils 2017-04-25T13:42:32.736-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:32.736-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:32.736-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:42:32.737-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
StatisticsManager 2017-04-25T13:42:32.738-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:42:32.738-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
XDCRFactory 2017-04-25T13:42:32.738-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:42:32.738-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716 is constructed, start it
GenericPipeline 2017-04-25T13:42:32.739-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[83 153 41 108 119 159 170 24 87 106 112 116 121 94 98 123 161 11 49 62 90 7 12 84 114 39 64 132 140 52 66 164 59 99 107 4 33 125 25 32 54 110 22 30 77 165 13 44 53 43 80 3 40 111 168 9 71 91 134 1 16 102 135 158 146 37 133 38 47 73 152 92 130 21 65 75 96 124 67 74 105 169 35 103 136 151 141 167 23 155 163 5 18 142 57 122 50 126 166 8 17 20 51 127 63 138 154 69 115 118 131 15 113 0 68 82 95 148 19 31 89 6 93 34 61 72 88 129 58 100 156 48 104 120 147 70 101 137 109 128 60 86 10 14 42 97 26 143 144 117 29 162 2 76 150 27 45 85 149 139 28 145 160 46 56 79 36 55 157 78 81] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[208 314 195 197 311 199 245 267 281 322 278 264 293 327 206 290 297 172 192 235 269 242 272 326 308 194 213 231 251 318 171 203 205 224 307 198 263 288 339 182 261 316 176 191 244 218 258 262 188 321 181 248 276 286 319 274 310 196 256 260 186 217 268 301 174 222 247 299 304 335 238 202 325 229 216 234 215 243 250 334 232 246 211 317 333 177 210 306 271 283 294 228 233 338 282 292 337 190 249 295 302 207 287 223 201 330 189 257 309 219 239 332 187 230 236 284 323 175 289 298 179 296 303 252 331 185 280 313 240 291 184 237 329 204 305 315 320 273 328 226 324 220 259 340 180 173 183 193 241 279 225 275 178 212 300 227 253 255 312 254 265 336 200 209 221 266 270 277 285 214]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[117 29 162 2 76 150 27 45 85 149 139 28 145 160 46 56 79 36 55 157 78 81 83 153 41 108 119 159 170 24 87 106 112 116 121 94 98 123 161 11 49 62 90 7 12 84 114 39 64 132 140 52 66 164 59 99 107 4 33 125 25 32 54 110 22 30 77 165 13 44 53 43 80 3 40 111 168 9 71 91 134 1 16 102 135 158 146 37 133 38 47 73 152 92 130 21 65 75 96 124 67 74 105 169 35 103 136 151 141 167 23 155 163 5 18 142 57 122 50 126 166 8 17 20 51 127 63 138 154 69 115 118 131 15 113 0 68 82 95 148 19 31 89 6 93 34 61 72 88 129 58 100 156 48 104 120 147 70 101 137 109 128 60 86 10 14 42 97 26 143 144] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[220 259 340 180 173 183 193 241 279 225 275 178 212 300 227 253 255 312 254 265 336 200 209 221 266 270 277 285 214 208 314 195 197 311 199 245 267 281 322 278 264 293 327 206 290 297 172 192 235 269 242 272 326 308 194 213 231 251 318 171 203 205 224 307 198 263 288 339 182 261 316 176 191 244 218 258 262 188 321 181 248 276 286 319 274 310 196 256 260 186 217 268 301 174 222 247 299 304 335 238 202 325 229 216 234 215 243 250 334 232 246 211 317 333 177 210 306 271 283 294 228 233 338 282 292 337 190 249 295 302 207 287 223 201 330 189 257 309 219 239 332 187 230 236 284 323 175 289 298 179 296 303 252 331 185 280 313 240 291 184 237 329 204 305 315 320 273 328 226 324]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 target_nozzle_per_node:2 replication_type:capi worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression:]
CheckpointManager 2017-04-25T13:42:32.740-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716...
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:32.752-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:42:32.753-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
CheckpointManager 2017-04-25T13:42:32.753-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:42:32.753-05:00 [INFO] Checkpointing starts in 1801.96 sec
PipelineRuntimeCtx 2017-04-25T13:42:32.753-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:42:32.753-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:42:32.753-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:42:32.758-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:42:32.758-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:42:32.758-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:42:32.758-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:32.758-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:42:32.758-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:42:32.758-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:42:32.760-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:42:32.762-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:42:32.790-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:42:32.790-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:42:32.790-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:42:32.803-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:32.803-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:32.803-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:42:32.812-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:32.812-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:42:32.823-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:42:32.823-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:42:32.827-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:42:32.842-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:42:32.842-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:32.843-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:42:32.843-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:42:32.844-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:42:32.845-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:42:32.873-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:42:32.873-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:42:32.876-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:42:32.896-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:42:32.896-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
AsyncListener 2017-04-25T13:42:32.989-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:42:32.989-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:42:32.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
CheckpointManager 2017-04-25T13:42:33.004-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:42:33.031-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:42:33.031-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:33.031-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:42:33.032-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:42:33.032-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:42:33.032-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:42:33.032-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716 is started----------
PipelineManager 2017-04-25T13:42:33.032-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:42:33.032-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:42:33.035-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:42:33.222-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:42:33.223-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:42:33.223-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:42:33.223-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:42:33 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:42:33.308-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:33.343-05:00 [INFO] Send batch count=1
2017/04/25 13:42:33 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:42:33.356-05:00 [INFO] Send batch count=102
CapiNozzle 2017-04-25T13:42:33.376-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:33.376-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:42:33 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 107 successful
CapiNozzle 2017-04-25T13:42:33.405-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:33.405-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:33.417-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:42:33 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 110 successful
CapiNozzle 2017-04-25T13:42:33.465-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:42:33 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:42:33.718-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:33.731-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:42:33 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 23 successful
StatisticsManager 2017-04-25T13:42:33.812-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:42:33 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 28 successful
CapiNozzle 2017-04-25T13:42:33.904-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:33.922-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:42:33.946-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:42:33 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 140 successful
CapiNozzle 2017-04-25T13:42:33.959-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:42:33 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:42:33 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 67 successful
CapiNozzle 2017-04-25T13:42:34.113-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:34.113-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:34.113-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:34.113-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:42:34.113-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:42:34.130-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:42:34.130-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:42:34.132-05:00 [INFO] err_list=[{"time":"2017-04-25T13:42:34.132903343-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:42:34.133-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:42:34 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 74 successful
CapiNozzle 2017-04-25T13:42:34.221-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:34.221-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:34.238-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
2017/04/25 13:42:34 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 83 successful
CheckpointManager 2017-04-25T13:42:34.416-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
CheckpointManager 2017-04-25T13:42:34.416-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:42:34.429-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:42:34 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:42:34 UPR_STREAMREQ for vb 95 successful
CapiNozzle 2017-04-25T13:42:34.481-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
DcpNozzle 2017-04-25T13:42:34.553-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:42:34.553-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:42:34.555-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:42:34.579-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
StatisticsManager 2017-04-25T13:42:34.769-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:42:34.769-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716 {"Errors": "[{\"time\":\"2017-04-25T13:42:34.132903343-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 353, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 17514, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 353, "set_filtered": 0, "set_received_from_dcp": 17514, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:42:34.984-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:34.984-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:42:34.984-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:42:34.984-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:34.984-05:00 [INFO] Send batch count=211
CapiNozzle 2017-04-25T13:42:35.024-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:35.024-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:35.037-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:42:35.249-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:42:35.267-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:42:35.362-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:35.376-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:42:35.627-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:42:35.679-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:42:35.679-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:42:35.679-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:35.679-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:35.700-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:35.724-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:42:35.755-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:35.755-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:35.781-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:42:36.074-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:36.074-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:42:36.074-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:42:36.074-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:36.074-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:36.157-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:36.157-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:36.183-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:42:36.428-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:42:36.449-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:42:36.954-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:36.979-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:42:37.298-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:42:37.313-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:42:37.735-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:37.780-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:42:37.918-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:42:37.918-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:42:37.918-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:42:37.918-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:37.918-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:38.021-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:38.021-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:38.037-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:42:38.137-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:42:38.137-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:42:38.137-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:42:38.137-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:38.137-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:38.253-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:38.254-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:38.270-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:42:39.453-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:42:39.454-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:39.501-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:42:39.520-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:42:40.049-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:40.074-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
PipelineManager 2017-04-25T13:42:40.251-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:34.132903343-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:42:40.315-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:42:40.335-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
ReplicationManager 2017-04-25T13:42:40.511-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:42:40.553-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:40.553-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:42:40.553-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:42:40.553-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:40.553-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:40.610-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:40.610-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:40.626-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:42:40.772-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:42:40.772-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:42:40.772-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:42:40.772-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:40.772-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:40.899-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:40.899-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:40.909-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:42:41.048-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:41.060-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:42:41.419-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:41.431-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:42:41.766-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:41.796-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:42:41.957-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:41.982-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:42:42.500-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:42.500-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:42:42.500-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:42:42.500-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:42.500-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:42.605-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:42.605-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:42.616-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:42:42.681-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:42.681-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:42:42.681-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:42:42.681-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:42.681-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:42.742-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:42.742-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:42.753-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
GenericSupervisor 2017-04-25T13:42:42.760-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:42:42.760-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:42:42.760-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:42:42.905-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:42:42.916-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
ToplogyChangeDetector 2017-04-25T13:42:43.056-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:42:43.059-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:42:43.069-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:42:43.297-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:42:43.308-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:42:43.453-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:42:43.464-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:42:43.611-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:42:43.611-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:42:43.611-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:42:43.611-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:43.612-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:43.677-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:43.677-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:43.688-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:42:43.812-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:42:43.812-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:42:43.812-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:42:43.812-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:43.812-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:43.877-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:43.877-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:43.888-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:42:44.016-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:42:44.027-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
PipelineManager 2017-04-25T13:42:44.133-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:42:44.133-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:42:44.133-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:44.133-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
PipelineRuntimeCtx 2017-04-25T13:42:44.133-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:42:44.133-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:44.133-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:42:44.133-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:44.133497178 -0500 CDT
GenericSupervisor 2017-04-25T13:42:44.133-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:44.133-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:42:44.133-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:44.133-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:42:44.133-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:44.133-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:42:44.133-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:42:44.133-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:42:44.133-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:42:44.133-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-671709716 has been stopped
GenericPipeline 2017-04-25T13:42:44.134-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-671709716 is stopped
PipelineManager 2017-04-25T13:42:44.134-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:42:44.134-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:34.132903343-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:42:44.134-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:44.134-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
GenericPipeline 2017-04-25T13:42:44.134-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
GenericPipeline 2017-04-25T13:42:44.134-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
GenericPipeline 2017-04-25T13:42:44.134-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-671709716
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:42:44.134-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.134-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:44.134-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:44.135-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:42:44.135-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:42:44.135-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:42:44.135-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:42:44.135-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:42:44.135-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:44.135-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:42:44.135-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:44.135150734 -0500 CDT
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:42:44.135-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:44.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:42:44.135-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:44.135483567 -0500 CDT
CapiNozzle 2017-04-25T13:42:44.136-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:42:44.136-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:44.136-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:42:44.136-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:42:44.136-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:44.136-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:42:44.136-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:42:44.136-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:42:44.136-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
AsyncListener 2017-04-25T13:42:44.136-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:42:44.136-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:44.136-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:42:44.138-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:44.138-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:44.138115195 -0500 CDT
DcpNozzle 2017-04-25T13:42:44.138-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:42:44.138-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:44.138-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:42:44.138-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:44.138-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:44.138419996 -0500 CDT
DcpNozzle 2017-04-25T13:42:44.138-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:42:44.138-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:44.138-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:42:44 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:42:44 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:42:44 Stream Closed for vb 1 StreamEnd simulated
XDCRFactory 2017-04-25T13:42:44.261-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:44.262-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:42:44.262-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:42:44.324-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:44.324-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:42:44.324-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:44.325-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:42:44.325-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:44.325-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:42:44.325-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:42:44.325-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:42:44.325-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
GenericSupervisor 2017-04-25T13:42:44.327-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:42:44.327-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
GenericSupervisor 2017-04-25T13:42:44.327-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:44.327-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:44.327-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:44.327-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:44.327-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:42:44.327-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
CheckpointManager 2017-04-25T13:42:44.327-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
CapiUtils 2017-04-25T13:42:44.449-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:44.449-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:44.449-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:42:44.450-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
StatisticsManager 2017-04-25T13:42:44.450-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:42:44.450-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
XDCRFactory 2017-04-25T13:42:44.450-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:42:44.451-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045 is constructed, start it
GenericPipeline 2017-04-25T13:42:44.452-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[130 164 44 90 116 14 45 28 165 107 50 55 147 65 83 99 21 39 96 122 100 109 124 10 40 115 29 82 167 54 59 110 166 17 34 38 37 101 152 11 32 56 9 70 145 68 78 97 146 49 52 61 135 160 161 36 1 22 30 5 23 81 20 46 105 125 140 6 66 92 131 84 88 111 89 91 139 63 77 119 42 43 158 41 76 95 13 73 86 33 144 67 98 132 18 47 48 114 155 170 60 128 31 102 156 26 19 71 126 94 157 118 121 4 58 112 74 123 150 69 106 133 148 159 12 108 127 120 151 72 87 149 75 85 113 62 143 154 168 162 15 134 138 57 103 35 0 2 8 169 27 137 142 163 7 51 153 25 136 24 53 129 64 117 16 79 104 141 3 80 93] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[200 229 274 188 218 222 250 279 171 291 339 301 307 318 256 209 191 206 303 249 212 183 196 230 287 246 290 309 182 202 180 261 204 214 173 220 236 248 275 278 192 328 197 213 305 235 247 257 335 179 216 253 268 269 289 294 321 284 292 242 227 240 233 207 224 251 300 312 187 195 203 239 243 271 189 228 327 281 263 276 298 326 184 299 331 265 293 175 185 322 338 194 231 254 273 317 329 330 325 178 260 267 272 308 315 304 193 221 313 172 210 285 336 244 288 181 323 311 226 252 333 340 337 177 225 190 296 302 324 232 217 198 211 234 283 297 208 262 316 334 174 176 205 286 306 270 295 310 199 201 223 219 245 264 280 320 282 332 277 314 237 258 215 186 238 259 266 241 255 319]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[75 85 113 62 143 154 168 15 134 138 162 57 103 0 2 8 35 169 27 137 142 7 51 153 163 25 24 53 129 136 64 117 16 79 3 80 93 104 141 44 90 116 130 164 14 45 28 165 107 50 55 147 65 83 99 21 39 96 122 100 109 10 40 115 124 29 82 167 166 17 34 38 54 59 110 37 101 11 32 56 152 9 70 145 146 49 52 61 68 78 97 135 160 161 1 22 30 36 5 23 81 20 46 6 66 92 105 125 140 84 88 111 131 89 91 63 77 119 139 42 43 41 76 95 158 13 73 86 33 144 18 47 48 67 98 132 114 155 170 60 128 31 102 156 26 19 71 126 94 157 118 121 4 58 112 74 123 150 69 106 12 108 127 133 148 159 120 72 87 149 151] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[337 177 225 302 324 190 296 217 232 234 283 198 211 297 334 174 176 208 262 316 205 286 306 199 201 223 270 295 310 219 245 264 280 320 282 332 277 314 237 215 258 238 259 186 266 241 255 319 229 274 200 250 279 171 188 218 222 291 339 256 301 307 318 209 191 206 303 249 183 196 212 230 287 182 246 290 309 202 180 261 204 214 248 275 173 220 236 278 192 197 213 305 328 335 235 247 257 253 268 179 216 269 289 284 292 294 321 227 240 242 207 233 224 251 300 312 187 195 271 189 203 239 243 228 327 281 298 326 263 276 184 299 331 175 265 293 185 322 338 194 231 329 330 254 273 317 178 260 325 267 272 308 304 315 193 221 313 336 172 210 285 181 244 288 323 226 252 311 333 340]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info stats_interval:1000 replication_type:capi active:true checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:42:44.452-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045...
AsyncListener 2017-04-25T13:42:44.460-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:44.461-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:42:44.461-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:44.461-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:42:44.461-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:42:44.461-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:42:44.461-05:00 [INFO] Checkpointing starts in 1804.751 sec
PipelineRuntimeCtx 2017-04-25T13:42:44.461-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:42:44.461-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:42:44.461-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:42:44.461-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:42:44.466-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:42:44.466-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:42:44.466-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:42:44.466-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:42:44.470-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:42:44.484-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:44.484-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:44.484-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:42:44.496-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:42:44.500-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:42:44.514-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:42:44.514-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:42:44.514-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:42:44.525-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:42:44.525-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:42:44.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:42:44.594-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:42:44.594-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:44.594-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:42:44.594-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:42:44.594-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:42:44.594-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:42:44.630-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:42:44.642-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:42:44.642-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:42:44.642-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:42:44.652-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:44.652-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:44.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:42:44.743-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:44.743-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:42:44.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:42:44.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:42:44.743-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:42:44.744-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045 is started----------
PipelineManager 2017-04-25T13:42:44.744-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:42:44.744-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:42:44.744-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:42:45.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:42:45.828-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:42:45.829-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:42:45.829-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:42:45.829-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:42:45 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:42:45.896-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:42:45.899-05:00 [INFO] Send batch count=109
CapiNozzle 2017-04-25T13:42:45.916-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:45.916-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:42:45 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:42:45.936-05:00 [INFO] Send batch count=7
CapiNozzle 2017-04-25T13:42:45.939-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:42:45.947-05:00 [ERROR] Received malformed response from tcp connection vb=300. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f2/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}

CapiNozzle 2017-04-25T13:42:45.947-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err Received malformed response from tcp connection vb=300. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f2/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}
.
2017/04/25 13:42:45 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:42:45 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:42:45.967-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:42:46 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 223 successful
CapiNozzle 2017-04-25T13:42:46.118-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:42:46 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 217 successful
CapiNozzle 2017-04-25T13:42:46.122-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:46.149-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:42:46.149-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:42:46 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 135 successful
CapiNozzle 2017-04-25T13:42:46.183-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:46.196-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:42:46 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 261 successful
CapiNozzle 2017-04-25T13:42:46.315-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:46.315-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:42:46.315-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:42:46.315-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:42:46.315-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
CapiNozzle 2017-04-25T13:42:46.319-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:46.336-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
PipelineManager 2017-04-25T13:42:46.348-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:42:46.348-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:42:46.353-05:00 [INFO] err_list=[{"time":"2017-04-25T13:42:46.35318147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:42:46.353-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:42:46 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 266 successful
CapiNozzle 2017-04-25T13:42:46.424-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:46.424-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:46.445-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
StatisticsManager 2017-04-25T13:42:46.471-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:42:46.472-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045 {"Errors": "[{\"time\":\"2017-04-25T13:42:46.35318147-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

2017/04/25 13:42:46 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 76 successful
CapiNozzle 2017-04-25T13:42:46.587-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:46.587-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:46.587-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:46.587-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:46.587-05:00 [INFO] Send batch count=500
2017/04/25 13:42:46 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 87 successful
CapiNozzle 2017-04-25T13:42:46.711-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:46.711-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:42:46 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 293 successful
CapiNozzle 2017-04-25T13:42:46.739-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:42:46 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 90 successful
CheckpointManager 2017-04-25T13:42:46.802-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
CheckpointManager 2017-04-25T13:42:46.802-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:42:46 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 97 successful
CapiNozzle 2017-04-25T13:42:46.876-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:46.888-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:42:46 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:42:46 UPR_STREAMREQ for vb 196 successful
DcpNozzle 2017-04-25T13:42:46.953-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:42:46.998-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:42:47.053-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:47.063-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:42:47.256-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:47.288-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:42:47.385-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:47.461-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:42:47.660-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:47.660-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:42:47.660-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:42:47.660-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:47.660-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:47.750-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:47.750-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:47.779-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:42:47.911-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:42:47.911-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:42:47.911-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:42:47.911-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:47.911-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:47.972-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:47.972-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:47.989-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:42:48.122-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:48.139-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:42:48.329-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:48.363-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:42:48.524-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:48.541-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:42:48.631-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:48.659-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:42:48.908-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:42:48.908-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:42:48.908-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:42:48.908-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:48.908-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:49.059-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:49.059-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:49.081-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:42:49.209-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:49.209-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:49.209-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:49.209-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:49.209-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:49.342-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:49.342-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:49.357-05:00 [INFO] Retrying update docs for vb 105 for the 1th time
CapiNozzle 2017-04-25T13:42:49.531-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:49.550-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:42:49.707-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:42:49.724-05:00 [INFO] Retrying update docs for vb 105 for the 2th time
CapiNozzle 2017-04-25T13:42:49.977-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:49.991-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:42:50.140-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:42:50.201-05:00 [INFO] Retrying update docs for vb 105 for the 3th time
CapiNozzle 2017-04-25T13:42:50.808-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:42:50.808-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:42:50.808-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:42:50.808-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:50.808-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:50.961-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:50.961-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:50.985-05:00 [ERROR] batchUpdateDocs for vb 105 failed with err receieved unexpected status code, 500, from update docs request for vb 105
.
CapiNozzle 2017-04-25T13:42:50.985-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 105 after 3 retries
CapiNozzle 2017-04-25T13:42:50.985-05:00 [ERROR] Raise error condition batch update docs failed for vb 105 after 3 retries
GenericSupervisor 2017-04-25T13:42:50.985-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:50.985-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:50.992-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:42:51.049-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:51.049-05:00 [ERROR] batchUpdateDocs for vb 109 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:51.067-05:00 [INFO] Retrying update docs for vb 109 for the 1th time
CapiNozzle 2017-04-25T13:42:51.392-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:51.415-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:42:51.601-05:00 [ERROR] batchUpdateDocs for vb 109 failed with err receieved unexpected status code, 500, from update docs request for vb 109
.
CapiNozzle 2017-04-25T13:42:51.619-05:00 [INFO] Retrying update docs for vb 109 for the 2th time
CapiNozzle 2017-04-25T13:42:51.836-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:51.860-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:42:51.944-05:00 [ERROR] batchUpdateDocs for vb 109 failed with err receieved unexpected status code, 500, from update docs request for vb 109
.
CapiNozzle 2017-04-25T13:42:51.958-05:00 [INFO] Retrying update docs for vb 109 for the 3th time
CapiNozzle 2017-04-25T13:42:52.318-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:42:52.318-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:42:52.318-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:42:52.318-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:52.318-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:52.388-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:52.388-05:00 [ERROR] batchUpdateDocs for vb 305 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:52.412-05:00 [INFO] Retrying update docs for vb 305 for the 1th time
CapiNozzle 2017-04-25T13:42:52.524-05:00 [ERROR] batchUpdateDocs for vb 109 failed with err receieved unexpected status code, 500, from update docs request for vb 109
.
CapiNozzle 2017-04-25T13:42:52.524-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 109 after 3 retries
CapiNozzle 2017-04-25T13:42:52.524-05:00 [ERROR] Raise error condition batch update docs failed for vb 109 after 3 retries
GenericSupervisor 2017-04-25T13:42:52.524-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:52.524-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:52.579-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:52.579-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:52.594-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:42:52.683-05:00 [ERROR] batchUpdateDocs for vb 305 failed with err receieved unexpected status code, 500, from update docs request for vb 305
.
CapiNozzle 2017-04-25T13:42:52.694-05:00 [INFO] Retrying update docs for vb 305 for the 2th time
CapiNozzle 2017-04-25T13:42:53.068-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:53.094-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:42:53.574-05:00 [ERROR] batchUpdateDocs for vb 305 failed with err receieved unexpected status code, 500, from update docs request for vb 305
.
CapiNozzle 2017-04-25T13:42:53.598-05:00 [INFO] Retrying update docs for vb 305 for the 3th time
CapiNozzle 2017-04-25T13:42:53.863-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:53.886-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:42:54.080-05:00 [ERROR] batchUpdateDocs for vb 305 failed with err receieved unexpected status code, 500, from update docs request for vb 305
.
CapiNozzle 2017-04-25T13:42:54.080-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 305 after 3 retries
CapiNozzle 2017-04-25T13:42:54.080-05:00 [ERROR] Raise error condition batch update docs failed for vb 305 after 3 retries
GenericSupervisor 2017-04-25T13:42:54.080-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:54.080-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:54.113-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 215
CapiNozzle 2017-04-25T13:42:54.185-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:54.185-05:00 [ERROR] batchUpdateDocs for vb 215 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:54.209-05:00 [INFO] Retrying update docs for vb 215 for the 1th time
CapiNozzle 2017-04-25T13:42:54.367-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:42:54.367-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:42:54.367-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:42:54.367-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:54.367-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:42:54.445-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:54.445-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:54.463-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
GenericSupervisor 2017-04-25T13:42:54.466-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:42:54.467-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:42:54.467-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:42:54.681-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:42:54.729-05:00 [ERROR] batchUpdateDocs for vb 215 failed with err receieved unexpected status code, 500, from update docs request for vb 215
.
CapiNozzle 2017-04-25T13:42:54.740-05:00 [INFO] Retrying update docs for vb 215 for the 2th time
CapiNozzle 2017-04-25T13:42:54.892-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:54.903-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:42:55.113-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:46.35318147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:42:55.206-05:00 [ERROR] batchUpdateDocs for vb 215 failed with err receieved unexpected status code, 500, from update docs request for vb 215
.
CapiNozzle 2017-04-25T13:42:55.217-05:00 [INFO] Retrying update docs for vb 215 for the 3th time
CapiNozzle 2017-04-25T13:42:55.392-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:42:55.403-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:42:56.325-05:00 [ERROR] batchUpdateDocs for vb 215 failed with err receieved unexpected status code, 500, from update docs request for vb 215
.
CapiNozzle 2017-04-25T13:42:56.325-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 215 after 3 retries
CapiNozzle 2017-04-25T13:42:56.325-05:00 [ERROR] Raise error condition batch update docs failed for vb 215 after 3 retries
GenericSupervisor 2017-04-25T13:42:56.325-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:56.325-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:42:56.353-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:42:56.353-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:42:56.353-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:56.353-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
PipelineRuntimeCtx 2017-04-25T13:42:56.353-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:42:56.353-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:56.353-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:42:56.353-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:42:56.353-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:42:56.353-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:42:56.353-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:42:56.353-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:42:56.353-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:56.353-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:42:56.353-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:56.353896401 -0500 CDT
GenericSupervisor 2017-04-25T13:42:56.354-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:56.354-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:42:56.354-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-325176045 has been stopped
GenericPipeline 2017-04-25T13:42:56.354-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-325176045 is stopped
PipelineManager 2017-04-25T13:42:56.354-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:42:56.354-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:46.35318147-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:42:56.354-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:42:56.354-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
GenericPipeline 2017-04-25T13:42:56.355-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
GenericPipeline 2017-04-25T13:42:56.355-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
GenericPipeline 2017-04-25T13:42:56.355-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-325176045
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:42:56.355-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:42:56.355-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:56.355-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:42:56.355-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:42:56.355-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:56.355-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:42:56.355-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:42:56.355-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:56.355-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:42:56.355-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:42:56.356-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:42:56.356-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:42:56.356-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:42:56.356-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:42:56.356-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:56.355932447 -0500 CDT
DcpNozzle 2017-04-25T13:42:56.356-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:42:56.356-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:56.356198295 -0500 CDT
CapiNozzle 2017-04-25T13:42:56.357-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:42:56.357-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:56.357-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:42:56.357-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:42:56.357-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:42:56.357-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:56.358-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
2017/04/25 13:42:56 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:42:56.363-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:56.363-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:56.363329095 -0500 CDT
DcpNozzle 2017-04-25T13:42:56.363-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:42:56.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:42:56 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:42:56 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:42:56 Stream Closed for vb 3 StreamEnd simulated
DcpNozzle 2017-04-25T13:42:56.363-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:42:56.363657343 -0500 CDT
GenericPipeline 2017-04-25T13:42:56.363-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:56.363-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:42:56.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:42:56.363-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:56.363-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:42:56 Stream Closed for vb 171 StreamEnd simulated
CapiNozzle 2017-04-25T13:42:56.433-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:42:56.433-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:42:56.433-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:42:56.433-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:42:56.501-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:56.501-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:42:56.501-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:42:56.590-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:42:56.591-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:42:56.591-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:56.591-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:42:56.591-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:42:56.591-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:42:56.591-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:42:56.592-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:42:56.592-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
GenericSupervisor 2017-04-25T13:42:56.594-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:42:56.594-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
GenericSupervisor 2017-04-25T13:42:56.594-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:56.594-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:56.594-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:56.594-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:42:56.594-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:42:56.594-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
CheckpointManager 2017-04-25T13:42:56.594-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
CapiUtils 2017-04-25T13:42:56.653-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:56.653-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:56.653-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:42:56.654-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
StatisticsManager 2017-04-25T13:42:56.654-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:42:56.654-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
XDCRFactory 2017-04-25T13:42:56.655-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:42:56.655-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974 is constructed, start it
GenericPipeline 2017-04-25T13:42:56.656-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[136 66 86 157 48 96 101 18 47 130 79 41 49 90 146 124 158 169 73 78 67 94 75 83 98 115 126 128 162 0 14 107 166 35 58 70 108 92 44 129 153 22 37 63 65 72 89 3 110 118 30 95 103 160 161 12 13 122 116 45 121 21 71 31 40 42 93 59 24 168 113 133 147 50 154 16 46 85 32 106 111 142 55 87 102 17 36 114 60 74 82 33 56 69 159 11 77 131 61 120 10 125 25 62 141 165 167 29 145 164 53 119 54 76 84 28 112 20 57 39 43 123 104 134 9 91 8 68 27 34 132 148 7 105 143 170 2 38 100 135 150 155 15 64 81 117 152 138 140 4 97 151 156 127 23 26 52 149 163 51 137 6 109 1 80 5 139 19 88 99 144] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[200 269 299 189 227 213 214 219 306 254 288 305 188 245 253 319 259 292 329 239 301 330 338 233 270 318 321 237 250 212 289 297 328 228 241 243 178 224 295 307 290 184 207 268 293 171 285 327 222 244 334 220 247 198 201 261 312 322 202 218 194 256 177 180 181 248 238 258 211 229 230 296 173 262 242 315 324 174 179 192 325 226 317 300 314 236 255 264 337 199 231 273 282 332 240 279 172 191 208 302 267 286 182 276 283 336 257 316 320 335 195 196 287 294 263 340 185 193 277 326 225 280 260 281 310 187 235 251 272 333 309 186 204 249 274 275 217 221 304 209 232 176 246 234 206 175 210 215 265 223 278 291 197 266 323 331 339 203 271 311 313 183 284 298 308 190 216 252 205 303]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[43 123 39 134 104 91 9 68 8 27 132 148 34 105 143 170 7 38 100 135 150 155 2 64 81 117 152 15 140 138 97 151 156 4 127 26 52 149 163 23 137 51 6 109 80 1 139 5 88 99 144 19 136 86 66 157 96 101 48 47 130 18 79 49 90 146 41 158 169 124 78 73 94 67 83 98 115 126 75 162 128 14 107 166 0 58 70 108 35 92 44 37 63 65 72 89 129 153 22 110 118 3 95 103 160 161 30 13 122 12 116 121 45 71 21 40 42 93 31 59 168 24 133 147 113 154 50 46 85 16 106 111 142 32 87 102 55 36 114 17 74 82 60 56 69 33 159 77 131 11 120 61 125 10 62 141 165 167 25 145 164 29 119 53 76 84 54 112 28 57 20] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[263 185 193 277 326 340 280 225 260 187 235 251 272 281 310 333 186 204 249 274 275 309 217 221 304 209 232 176 234 246 206 175 210 215 265 223 278 291 197 266 323 331 339 203 271 311 313 183 284 298 308 190 216 252 205 303 200 269 299 189 227 213 214 219 306 254 288 305 188 245 253 319 259 292 329 239 301 330 338 233 270 318 321 212 237 250 297 289 328 228 178 224 241 243 295 307 290 207 268 293 184 171 285 327 222 244 334 220 247 198 201 261 312 322 202 218 177 180 181 194 256 248 211 229 230 238 258 296 173 262 242 174 179 192 315 324 325 226 317 236 255 264 300 314 337 199 231 273 282 332 172 191 208 240 279 302 267 286 182 276 257 283 336 316 320 335 195 196 287 294]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[filter_expression: active:true worker_batch_size:500 failure_restart_interval:10 log_level:Info replication_type:capi doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000 checkpoint_interval:1800]
CheckpointManager 2017-04-25T13:42:56.656-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974...
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:42:56.673-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:42:56.673-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:42:56.673-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:42:56.673-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:42:56.673-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:42:56.673-05:00 [INFO] Checkpointing starts in 1802.975 sec
PipelineRuntimeCtx 2017-04-25T13:42:56.673-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:42:56.673-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:42:56.674-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:42:56.674-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:42:56.675-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:42:56.675-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:42:56.675-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:42:56.675-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:42:56.675-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:42:56.699-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:42:56.699-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:42:56.699-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:42:56.719-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:42:56.719-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:42:56.723-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:42:56.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:42:56.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:42:56.773-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:56.773-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:42:56.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:42:56.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:42:56.773-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:42:56.799-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:42:56.799-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:42:56.799-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:42:56.813-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:42:56.835-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:42:56.835-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:42:56.855-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:42:56.855-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:42:56.856-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:42:56.955-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:42:57.018-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:42:57.018-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:42:57.018-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:42:57.018-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:42:57.018-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:42:57.019-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:42:57.019-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974 is started----------
PipelineManager 2017-04-25T13:42:57.019-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:42:57.019-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:42:57.019-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:42:57.163-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:42:57.163-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:42:57.164-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:42:57.164-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:42:57 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:42:57.293-05:00 [INFO] Send batch count=67
CapiNozzle 2017-04-25T13:42:57.294-05:00 [INFO] Send batch count=500
2017/04/25 13:42:57 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:42:57.319-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:57.319-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:42:57 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 203 successful
CapiNozzle 2017-04-25T13:42:57.358-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:42:57 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:42:57.440-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:57.440-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:42:57 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 310 successful
CapiNozzle 2017-04-25T13:42:57.460-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:42:57 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 26 successful
CapiNozzle 2017-04-25T13:42:57.695-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:57.712-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
StatisticsManager 2017-04-25T13:42:57.736-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:42:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 140 successful
CapiNozzle 2017-04-25T13:42:57.919-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:42:57 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:42:57.949-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:42:57 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:42:57 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 292 successful
CheckpointManager 2017-04-25T13:42:58.358-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
CheckpointManager 2017-04-25T13:42:58.358-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:42:58 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:42:58 UPR_STREAMREQ for vb 197 successful
CapiNozzle 2017-04-25T13:42:58.442-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:58.460-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
DcpNozzle 2017-04-25T13:42:58.479-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:42:58.525-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:42:58.730-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:42:58.843-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:58.855-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:42:59.100-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:42:59.100-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:42:59.100-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:42:59.101-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:42:59.101-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:42:59.111-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:42:59.121-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:42:59.124-05:00 [INFO] err_list=[{"time":"2017-04-25T13:42:59.124487929-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:42:59.124-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:42:59.228-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:59.228-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:42:59.247-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:42:59.658-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:42:59.658-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:42:59.658-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:42:59.658-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:42:59.658-05:00 [INFO] Send batch count=172
CapiNozzle 2017-04-25T13:42:59.687-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:42:59.688-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:42:59.689-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:42:59.689-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974 {"Errors": "[{\"time\":\"2017-04-25T13:42:59.124487929-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 347, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 61109, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 347, "set_filtered": 0, "set_received_from_dcp": 61109, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:42:59.709-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:43:00.019-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:00.031-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:43:00.501-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:00.520-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:43:00.978-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:00.994-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:43:01.347-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:01.372-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:43:01.926-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:01.926-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:43:01.926-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:43:01.926-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:01.926-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:03.790-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:03.790-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:43:03.790-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:43:03.790-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:03.790-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:03.878-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:03.881-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:03.907-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:43:03.995-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:03.995-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:04.023-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:43:04.742-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:04.771-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:43:04.937-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:04.964-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:43:05.561-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:05.587-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:43:05.767-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:05.782-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:43:06.613-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:06.613-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:43:06.613-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:43:06.613-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:06.613-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:43:06.677-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:43:06.677-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:43:06.677-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:43:06.759-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:06.760-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:06.772-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:43:06.882-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:06.883-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:43:06.883-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:43:06.883-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:06.883-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:06.950-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:06.950-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
ToplogyChangeDetector 2017-04-25T13:43:06.951-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:43:06.961-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:43:07.280-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:43:07.291-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:43:07.497-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:43:07.508-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:43:07.867-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:43:07.878-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:43:08.100-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:43:08.110-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:43:08.429-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:43:08.429-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:43:08.429-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:43:08.429-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:08.429-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:08.513-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:08.513-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:08.523-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:43:08.564-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:43:08.564-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:43:08.564-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:43:08.564-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:08.564-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:08.633-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:08.633-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:08.643-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:43:08.890-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:43:08.901-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
PipelineManager 2017-04-25T13:43:09.124-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:43:09.124-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:43:09.124-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:43:09.125-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
PipelineRuntimeCtx 2017-04-25T13:43:09.125-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:43:09.125-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:09.125-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:43:09.125-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:09.125198937 -0500 CDT
GenericSupervisor 2017-04-25T13:43:09.125-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:09.125-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:43:09.125-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:43:09.125-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:43:09.125-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:43:09.125-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:43:09.125-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:43:09.125-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:43:09.125-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:43:09.125-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-591670974 has been stopped
GenericPipeline 2017-04-25T13:43:09.125-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-591670974 is stopped
PipelineManager 2017-04-25T13:43:09.125-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:43:09.125-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:42:59.124487929-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:43:09.125-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:43:09.126-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:43:09.126-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
GenericPipeline 2017-04-25T13:43:09.126-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:43:09.126-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-591670974
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:09.126-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.126-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:43:09.126-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:43:09.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:43:09.127-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:43:09.127-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:43:09.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:43:09.127-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:43:09.127-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericPipeline 2017-04-25T13:43:09.127-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:43:09.127-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:43:09.127-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:09.127-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:09.127-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:43:09.127-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:09.127502181 -0500 CDT
CapiNozzle 2017-04-25T13:43:09.127-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:09.127197627 -0500 CDT
DcpNozzle 2017-04-25T13:43:09.127-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:43:09.128-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:43:09.128-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:43:09.128-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:43:09.128-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:43:09.128-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:43:09.128-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:09.128-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:43:09.128-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:43:09.128-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:43:09.128-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:09.128-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:43:09.129-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:43:09.130-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:09.129992555 -0500 CDT
DcpNozzle 2017-04-25T13:43:09.130-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:43:09.130-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:09.130-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:43:09.130-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:43:09.130-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:09.13019677 -0500 CDT
DcpNozzle 2017-04-25T13:43:09.130-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:43:09.130-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:09.130-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:43:09 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:43:09 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:43:09.266-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:43:09.267-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:43:09.267-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:43:09.338-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:43:09.339-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:43:09.339-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:43:09.339-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:43:09.339-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:43:09.339-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:43:09.339-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:43:09.340-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:43:09.340-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
GenericSupervisor 2017-04-25T13:43:09.342-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:43:09.342-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
GenericSupervisor 2017-04-25T13:43:09.342-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:09.342-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:09.342-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:09.342-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:09.342-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:43:09.342-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
CheckpointManager 2017-04-25T13:43:09.342-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
CapiUtils 2017-04-25T13:43:09.410-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:43:09.411-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:43:09.411-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:43:09.411-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
StatisticsManager 2017-04-25T13:43:09.412-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:43:09.412-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
XDCRFactory 2017-04-25T13:43:09.412-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:43:09.412-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401 is constructed, start it
GenericPipeline 2017-04-25T13:43:09.413-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[52 98 27 81 124 14 16 100 131 144 157 42 71 92 104 122 95 28 33 107 155 156 46 47 72 3 80 55 1 74 79 31 145 151 56 86 117 7 35 20 62 143 13 36 83 164 142 23 40 44 54 127 18 29 48 70 110 39 68 89 153 123 136 158 12 64 103 163 65 165 159 128 67 93 170 73 88 169 76 134 63 126 141 148 24 119 112 9 105 2 102 129 6 30 50 121 116 152 53 58 87 45 49 75 120 94 0 34 90 5 15 115 149 161 69 162 114 167 51 57 108 160 166 10 99 139 135 19 59 60 84 111 32 106 17 125 22 118 130 150 154 41 61 96 4 66 91 101 137 38 97 133 140 146 25 132 11 77 82 113 21 37 78 138 147 43 85 109 168 8 26] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[260 281 280 317 179 247 337 266 270 284 232 261 325 208 235 306 193 315 338 191 251 265 189 175 243 186 215 181 302 327 177 184 180 269 310 329 291 314 276 256 330 195 248 309 313 203 240 242 289 335 283 285 211 245 290 196 296 217 173 201 254 272 298 308 263 318 252 223 212 274 187 172 200 328 336 312 264 282 292 194 239 323 331 199 339 207 228 333 241 210 244 259 178 300 176 222 297 174 219 206 324 301 236 262 278 326 234 198 238 224 233 249 295 182 185 304 214 253 257 273 294 192 237 197 213 220 225 267 320 332 311 319 321 227 231 279 204 316 250 271 171 268 299 229 255 221 340 183 226 307 188 190 288 305 202 286 205 275 277 293 303 209 218 230 246 258 287 322 334 216]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[134 141 148 63 126 24 119 112 9 105 129 2 102 50 121 6 30 116 152 87 53 58 75 120 45 49 94 90 0 34 115 149 161 5 15 69 162 114 108 160 166 167 51 57 139 10 99 135 60 84 111 19 59 32 106 17 125 130 22 118 150 154 96 41 61 91 101 137 4 66 133 140 146 38 97 25 132 82 11 77 113 78 138 147 21 37 109 168 43 85 8 26 52 98 124 27 81 14 16 100 131 144 92 104 122 157 42 71 95 28 33 156 107 155 72 46 47 3 80 55 79 1 74 151 31 145 117 56 86 7 35 143 20 62 83 164 13 36 44 54 127 142 23 40 48 70 110 18 29 89 153 39 68 123 136 158 103 163 12 64 65 165 159 128 170 67 93 169 73 88 76] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[333 241 210 244 259 178 176 222 297 300 174 206 219 324 236 262 301 234 278 326 198 238 224 233 249 295 182 185 257 273 294 304 214 253 192 237 197 213 220 225 332 267 320 227 231 279 311 319 321 204 316 250 271 171 268 299 229 255 221 340 183 226 307 188 190 288 202 286 305 277 293 303 205 275 218 209 230 246 258 287 322 216 334 260 281 179 247 280 317 266 270 284 337 261 325 232 208 235 306 193 315 338 251 265 191 189 175 186 215 243 181 302 327 177 184 269 310 329 180 291 314 276 195 248 256 330 203 240 242 309 313 283 285 289 335 196 211 245 290 296 217 173 201 254 308 272 298 252 263 318 223 212 187 274 172 328 336 200 312 194 239 264 282 292 323 331 199 339 228 207]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: checkpoint_interval:1800 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000 active:true worker_batch_size:500 doc_batch_size_kb:2048 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:43:09.413-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401...
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:09.424-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:43:09.424-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:09.424-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:43:09.424-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:43:09.424-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:43:09.424-05:00 [INFO] Checkpointing starts in 1804.618 sec
PipelineRuntimeCtx 2017-04-25T13:43:09.424-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:43:09.424-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:43:09.424-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:43:09.424-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:43:09.426-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:43:09.426-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:43:09.426-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:43:09.426-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:43:09.427-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:43:09.458-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:43:09.458-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:43:09.459-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:43:09.481-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:43:09.481-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:43:09.487-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:43:09.491-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:43:09.491-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:43:09.491-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:43:09.504-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:43:09.509-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:43:09.523-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:43:09.523-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:43:09.523-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:43:09.523-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:43:09.523-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:43:09.523-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:43:09.548-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:43:09.548-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:43:09.548-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:43:09.561-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:43:09.561-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:43:09.657-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:43:09.691-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:43:09.691-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:43:09.691-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:43:09.711-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:43:09.711-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:43:09.711-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:43:09.711-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401 is started----------
PipelineManager 2017-04-25T13:43:09.711-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:43:09.711-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:43:09.711-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:43:09.853-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:43:09.854-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:43:09.854-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:43:09.854-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:43:09 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:43:09 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:43:10.005-05:00 [INFO] Send batch count=73
CapiNozzle 2017-04-25T13:43:10.005-05:00 [INFO] Send batch count=140
2017/04/25 13:43:10 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:43:10.049-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:10.049-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:10.051-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:10.051-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:43:10 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 6 successful
CapiNozzle 2017-04-25T13:43:10.067-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:43:10.073-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
2017/04/25 13:43:10 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 117 successful
PipelineManager 2017-04-25T13:43:10.250-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:43:10 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:43:10.381-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:43:10.405-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
2017/04/25 13:43:10 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 132 successful
StatisticsManager 2017-04-25T13:43:10.479-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:43:10 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 246 successful
CapiNozzle 2017-04-25T13:43:10.518-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
2017/04/25 13:43:10 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 37 successful
CapiNozzle 2017-04-25T13:43:10.545-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:43:10.609-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
2017/04/25 13:43:10 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 249 successful
CapiNozzle 2017-04-25T13:43:10.637-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
2017/04/25 13:43:10 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 61 successful
CapiNozzle 2017-04-25T13:43:10.843-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:43:10.875-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
2017/04/25 13:43:10 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 77 successful
CapiNozzle 2017-04-25T13:43:10.936-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:43:10.936-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:43:10.936-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:43:10.936-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries]
ReplicationManager 2017-04-25T13:43:10.936-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries]
2017/04/25 13:43:10 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:43:10 UPR_STREAMREQ for vb 86 successful
PipelineManager 2017-04-25T13:43:10.957-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:43:10.957-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:43:10.961-05:00 [INFO] err_list=[{"time":"2017-04-25T13:43:10.961644141-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries"}]
PipelineManager 2017-04-25T13:43:10.961-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:43:11.011-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:11.011-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:43:11 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 296 successful
CapiNozzle 2017-04-25T13:43:11.039-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CheckpointManager 2017-04-25T13:43:11.052-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
CheckpointManager 2017-04-25T13:43:11.052-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:43:11 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:43:11 UPR_STREAMREQ for vb 92 successful
DcpNozzle 2017-04-25T13:43:11.121-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:43:11.131-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:43:11.235-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:43:11.235-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:43:11.235-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:43:11.235-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:11.235-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:11.326-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:11.326-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:11.347-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
StatisticsManager 2017-04-25T13:43:11.432-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:43:11.433-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401 {"Errors": "[{\"time\":\"2017-04-25T13:43:10.961644141-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 89, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 20321, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 89, "set_filtered": 0, "set_received_from_dcp": 20321, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:43:11.519-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:43:11.535-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:43:11.717-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:43:11.732-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:43:11.962-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:43:11.976-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:43:12.069-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:43:12.083-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:43:12.239-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:43:12.239-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:43:12.239-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:43:12.239-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:12.239-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:12.303-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:12.303-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:12.339-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:43:12.361-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:43:12.361-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:43:12.361-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:43:12.361-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:12.361-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:12.416-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:12.416-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:12.437-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:43:12.728-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:43:12.745-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:43:12.847-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:43:12.864-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:43:13.177-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:43:13.196-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:43:13.438-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:43:13.486-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:43:13.710-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:43:13.710-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:43:13.710-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:43:13.710-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:13.711-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:13.788-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:13.788-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:13.808-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:43:14.302-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:43:14.302-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:43:14.302-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:43:14.302-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:14.302-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:14.404-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:14.404-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:14.431-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:43:14.678-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:43:14.701-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:43:15.219-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:15.235-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:43:15.379-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:43:15.428-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:43:15.884-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:15.903-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:43:16.066-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:43:16.066-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:43:16.066-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:43:16.066-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:16.066-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:16.154-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:16.154-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:16.176-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:43:16.226-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:16.226-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:43:16.226-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:43:16.226-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:16.226-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:16.301-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:16.301-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:16.318-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:43:16.745-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:16.782-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:43:16.986-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:43:17.006-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:43:17.205-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:17.218-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:43:17.343-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:43:17.354-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:43:17.571-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:17.571-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:43:17.572-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:43:17.572-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:17.572-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:17.656-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:17.656-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:17.667-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:43:17.692-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:43:17.692-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:43:17.692-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:43:17.692-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:17.692-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:17.766-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:17.766-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:17.777-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:43:18.052-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:43:18.063-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:43:18.343-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:43:18.354-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:43:18.589-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:43:18.600-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:43:19.060-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:43:19.070-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:43:19.413-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:43:19.413-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:43:19.413-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:43:19.413-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:19.413-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:43:19.425-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:43:19.425-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:43:19.425-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:43:19.497-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:19.497-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:19.508-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:43:19.630-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:43:19.630-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:43:19.630-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:43:19.630-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:19.630-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:19.701-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:19.701-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
ToplogyChangeDetector 2017-04-25T13:43:19.712-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:43:19.712-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:43:19.781-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:43:19.792-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:43:20.126-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:43:20.137-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:43:20.295-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:43:20.305-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:43:20.520-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:43:20.531-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:43:20.785-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:43:20.785-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:43:20.785-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:43:20.785-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:20.785-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:20.878-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:20.878-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:20.889-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
PipelineManager 2017-04-25T13:43:20.961-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries 
PipelineManager 2017-04-25T13:43:20.962-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:43:20.962-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:43:20.962-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
PipelineRuntimeCtx 2017-04-25T13:43:20.962-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:43:20.962-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:43:20.962-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:20.962-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:43:20.962-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:20.962165847 -0500 CDT
GenericSupervisor 2017-04-25T13:43:20.962-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:20.962-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:43:20.962-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:43:20.962-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:43:20.962-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:43:20.962-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:43:20.962-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:43:20.962-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:43:20.962-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-339947401 has been stopped
GenericPipeline 2017-04-25T13:43:20.962-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-339947401 is stopped
PipelineManager 2017-04-25T13:43:20.962-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:43:20.962-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:43:10.961644141-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 303 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:43:20.962-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:43:20.963-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
GenericPipeline 2017-04-25T13:43:20.963-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
GenericPipeline 2017-04-25T13:43:20.963-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
GenericPipeline 2017-04-25T13:43:20.963-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-339947401
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:20.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:43:20.963-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:43:20.963-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:43:20.963-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:20.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:43:20.964-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:43:20.964-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:43:20.964-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:20.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:43:20.964-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:20.964-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:43:20.964-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:20.963994671 -0500 CDT
DcpNozzle 2017-04-25T13:43:20.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:43:20.964-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:20.964337789 -0500 CDT
DcpNozzle 2017-04-25T13:43:20.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:43:20.965-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:43:20.965-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:43:20.966-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:43:20.966-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:43:20.966-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:43:20.966-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:20.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:20.966-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:43:20.966-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:20.967-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
2017/04/25 13:43:20 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:43:20.976-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:43:20.976-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:20.976121707 -0500 CDT
DcpNozzle 2017-04-25T13:43:20.976-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:43:20.976-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:20.976-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:43:20 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:43:20.979-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:43:20.979-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:20.979535218 -0500 CDT
DcpNozzle 2017-04-25T13:43:20.979-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:43:20.979-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:20.979-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
XDCRFactory 2017-04-25T13:43:21.076-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:43:21.077-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:43:21.077-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:43:21.145-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:43:21.145-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:43:21.146-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:43:21.146-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:43:21.146-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:43:21.146-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:43:21.146-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:43:21.146-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:43:21.147-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
GenericSupervisor 2017-04-25T13:43:21.149-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:43:21.149-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
GenericSupervisor 2017-04-25T13:43:21.149-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:21.149-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:21.149-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:21.149-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:21.149-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:43:21.149-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
CheckpointManager 2017-04-25T13:43:21.149-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
CapiUtils 2017-04-25T13:43:21.193-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:43:21.193-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:43:21.193-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:43:21.194-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
StatisticsManager 2017-04-25T13:43:21.194-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:43:21.194-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
XDCRFactory 2017-04-25T13:43:21.195-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:43:21.195-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521 is constructed, start it
GenericPipeline 2017-04-25T13:43:21.196-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[52 100 84 12 36 47 0 35 119 128 154 48 167 168 2 10 26 50 148 160 9 74 111 138 155 76 109 135 159 164 43 131 133 158 78 140 141 64 96 30 75 80 87 165 15 23 33 79 110 151 14 137 42 44 86 97 157 1 6 27 65 123 60 41 88 103 38 68 146 147 170 7 94 112 5 28 98 124 73 16 108 3 34 71 101 126 145 81 55 120 166 49 62 83 156 161 91 45 95 13 32 104 22 51 150 162 105 121 17 39 114 117 29 53 118 127 142 18 56 58 77 82 99 21 70 37 40 61 125 129 130 11 122 19 54 69 115 163 143 169 24 25 136 144 31 57 152 134 8 20 89 139 4 59 63 67 90 116 85 106 113 66 92 149 46 93 132 153 72 102 107] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[280 312 243 211 230 238 252 302 201 276 330 263 291 317 237 283 220 233 288 326 206 214 301 319 332 248 221 249 250 256 321 329 273 277 320 339 340 179 231 255 259 295 337 183 225 275 200 217 234 290 191 251 271 331 174 227 264 178 261 278 176 228 303 195 208 265 241 318 212 324 180 190 194 202 296 196 181 203 229 186 218 236 336 213 257 284 199 209 232 322 258 269 298 244 274 216 260 205 245 267 270 328 189 281 247 224 193 262 333 334 177 207 219 223 292 300 182 254 266 272 286 294 204 310 215 239 246 282 299 287 304 210 325 327 240 289 187 309 314 185 279 315 335 172 285 307 175 222 305 306 192 323 171 235 197 198 226 242 293 188 316 253 311 173 184 338 268 297 308 313]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[273 277 320 339 340 179 231 255 259 295 337 183 225 275 200 217 234 290 191 251 271 331 174 227 264 178 261 278 176 228 303 195 208 265 241 318 212 324 180 190 194 202 296 196 181 203 229 186 218 236 336 213 257 284 199 209 232 322 258 269 298 244 274 216 260 205 245 267 270 328 189 281 247 224 193 262 333 334 177 207 219 223 292 300 182 254 266 272 286 294 204 310 215 239 246 282 299 287 304 210 325 327 240 289 187 309 314 185 279 315 335 172 285 307 175 222 305 306 192 323 171 235 197 198 226 242 293 188 316 253 311 173 184 338 268 297 308 313 280 312 243 211 230 238 252 302 201 276 330 263 291 317 237 283 220 233 288 326 206 214 301 319 332 248 221 249 250 256 321 329] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[64 96 30 75 80 87 165 15 23 33 79 110 151 14 137 42 44 86 97 157 1 6 27 65 123 60 41 88 103 38 68 146 147 170 7 94 112 5 28 98 124 73 16 108 3 34 71 101 126 145 81 55 120 166 49 62 83 156 161 91 45 95 13 32 104 22 51 150 162 105 121 17 39 114 117 29 53 118 127 142 18 56 58 77 82 99 21 70 37 40 61 125 129 130 11 122 19 54 69 115 163 143 169 24 25 136 144 31 57 152 134 8 20 89 139 4 59 63 67 90 116 85 106 113 66 92 149 46 93 132 153 72 102 107 52 100 84 12 36 47 0 35 119 128 154 48 167 168 2 10 26 50 148 160 9 74 111 138 155 76 109 135 159 164 43 131 133 158 78 140 141]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 failure_restart_interval:10 log_level:Info replication_type:capi checkpoint_interval:1800 worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000 filter_expression: active:true]
CheckpointManager 2017-04-25T13:43:21.196-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521...
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:21.204-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:43:21.204-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:21.204-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:43:21.204-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:43:21.204-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:43:21.204-05:00 [INFO] Checkpointing starts in 1800.191 sec
PipelineRuntimeCtx 2017-04-25T13:43:21.204-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:43:21.204-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:43:21.205-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:43:21.205-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:43:21.209-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:43:21.209-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:43:21.209-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:43:21.209-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:43:21.210-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:43:21.240-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:43:21.241-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:43:21.241-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:43:21.244-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:43:21.246-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:43:21.246-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:43:21.246-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:43:21.248-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:43:21.248-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:43:21.248-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:43:21.301-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:43:21.301-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:43:21.301-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:43:21.302-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:43:21.302-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:43:21.302-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:43:21.346-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:43:21.346-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
StatisticsManager 2017-04-25T13:43:21.359-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:43:21.360-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:43:21.361-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:43:21.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:43:21.400-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:43:21.477-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:43:21.477-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:43:21.477-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:43:21.477-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:43:21.477-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:43:21.477-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:43:21.477-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521 is started----------
PipelineManager 2017-04-25T13:43:21.478-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:43:21.478-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:43:21.478-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:43:21.798-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:43:21.799-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:43:21.799-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:43:21.799-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:43:21 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:43:21.885-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:43:21.906-05:00 [INFO] Send batch count=191
2017/04/25 13:43:21 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:43:21.909-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:43:21.929-05:00 [INFO] Send batch count=119
2017/04/25 13:43:21 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 4 successful
CapiNozzle 2017-04-25T13:43:21.956-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:21.956-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:21.968-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:21.968-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:43:21 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:43:21 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:43:21.991-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:43:22.008-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:43:22 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 115 successful
StatisticsManager 2017-04-25T13:43:22.262-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=5, changes_left=315503
CapiNozzle 2017-04-25T13:43:22.274-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:43:22 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:43:22.309-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:43:22 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 127 successful
CapiNozzle 2017-04-25T13:43:22.363-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:43:22 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 230 successful
CapiNozzle 2017-04-25T13:43:22.392-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:43:22 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 146 successful
CapiNozzle 2017-04-25T13:43:22.550-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:22.566-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:43:22 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 260 successful
CapiNozzle 2017-04-25T13:43:22.621-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:43:22 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 149 successful
CapiNozzle 2017-04-25T13:43:22.644-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:43:22 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 84 successful
CapiNozzle 2017-04-25T13:43:22.848-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:22.848-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:43:22.848-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:43:22.848-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:43:22.848-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:43:22.858-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:43:22.858-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:43:22.861-05:00 [INFO] err_list=[{"time":"2017-04-25T13:43:22.861335013-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:43:22.861-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:43:22 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 291 successful
CheckpointManager 2017-04-25T13:43:22.903-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
CheckpointManager 2017-04-25T13:43:22.903-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:43:22 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:43:22 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:43:22.936-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:22.936-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:22.936-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:22.936-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:43:22.936-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:43:22.936-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:22.936-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:22.959-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
DcpNozzle 2017-04-25T13:43:23.006-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:43:23.014-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:23.014-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:43:23 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:43:23 UPR_STREAMREQ for vb 296 successful
CapiNozzle 2017-04-25T13:43:23.033-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
DcpNozzle 2017-04-25T13:43:23.094-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:43:23.211-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:43:23.212-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521 {"Errors": "[{\"time\":\"2017-04-25T13:43:22.861335013-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315503, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 131, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 5, "docs_received_from_dcp": 12383, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 131, "set_filtered": 0, "set_received_from_dcp": 12383, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:43:23.313-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:23.344-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:43:23.492-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:23.507-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:43:23.731-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:23.750-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:43:23.892-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:23.913-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:43:24.115-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:24.116-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:43:24.116-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:43:24.116-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:24.116-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:24.190-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:24.192-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:24.221-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:43:24.301-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:43:24.301-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:43:24.301-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:43:24.301-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:24.301-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:43:24.472-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:24.472-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:24.504-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:43:24.516-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:24.557-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:43:24.953-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:43:24.967-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
PipelineManager 2017-04-25T13:43:25.186-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:43:22.861335013-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:43:25.329-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:25.364-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:43:25.622-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:43:25.648-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:43:26.380-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:43:26.386-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:43:26.386-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:43:26.389-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:43:26.389-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:43:26.389-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:43:26.386-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:26.394-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:43:26.389-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:26.418-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:43:34.467-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:43:34.480-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:43:34.480-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:43:34.480-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:43:34.480-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
PipelineRuntimeCtx 2017-04-25T13:43:34.480-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:43:34.480-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:34.482-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:43:34.482-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:43:34.483-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:43:34.483-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:34.480299916 -0500 CDT
GenericSupervisor 2017-04-25T13:43:34.483-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:43:34.483-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:43:34.483-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:43:34.483-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:43:34.483-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:43:34.483-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:43:34.483-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:43:34.483-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:43:34.483-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:43:34.483-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:43:34.483-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-146326521 has been stopped
GenericPipeline 2017-04-25T13:43:34.483-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-146326521 is stopped
PipelineManager 2017-04-25T13:43:34.483-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:43:34.483-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:43:22.861335013-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:43:34.484-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:43:34.484-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
GenericPipeline 2017-04-25T13:43:34.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
GenericPipeline 2017-04-25T13:43:34.484-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
GenericPipeline 2017-04-25T13:43:34.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-146326521
AsyncListener 2017-04-25T13:43:34.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:34.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:34.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:34.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:43:34.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:43:34.485-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:34.485-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:43:34.485-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:43:34.486-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:43:34.486-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:43:34.486-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:43:34.486-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:34.486-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:43:34.486-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:43:34.486-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:43:34.486-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:43:34.486-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:43:34.486-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:43:34.507-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:34.507-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:43:34.507-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:34.486260916 -0500 CDT
DcpNozzle 2017-04-25T13:43:34.507-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:43:34.507-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:34.486570621 -0500 CDT
CapiNozzle 2017-04-25T13:43:34.544-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:43:34.544-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.545-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:43:34.574-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:43:34.574-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:43:34.588-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
DcpNozzle 2017-04-25T13:43:34.802-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:43:34.802-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:34.802868083 -0500 CDT
DcpNozzle 2017-04-25T13:43:34.803-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:43:34.803-05:00 [INFO] Ask uprfeed to close
GenericPipeline 2017-04-25T13:43:34.803-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:34.803-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:43:34.803-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:43:34.803097966 -0500 CDT
DcpNozzle 2017-04-25T13:43:34.803-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:43:34.803-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:34.803-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:43:34.766-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:43:34.803-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:43:34.803-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:43:34.923-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:43:35.322-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:43:35.256-05:00 [INFO] Executing 0x793f70 timed out
CapiNozzle 2017-04-25T13:43:37.017-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:43:37.365-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:43:37.403-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:43:37.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
GenericPipeline 2017-04-25T13:43:35.897-05:00 [INFO] Executing 0x65e930 timed out
GenericPipeline 2017-04-25T13:43:37.850-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:43:37.870-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
GenericPipeline 2017-04-25T13:43:37.892-05:00 [INFO] Failed to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0, err=Execution timed out, let it alone to die
GenericPipeline 2017-04-25T13:43:37.892-05:00 [INFO] Source nozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 failed to stop in time, left it alone to die.
XDCRFactory 2017-04-25T13:43:39.730-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:43:39.730-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:43:39.744-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:43:39.858-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:43:39.898-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:43:39.913-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:43:39.914-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:43:39.914-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:43:39.914-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:43:39.926-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:43:39.944-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:43:39.951-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
GenericSupervisor 2017-04-25T13:43:39.953-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:43:39.953-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
GenericSupervisor 2017-04-25T13:43:39.953-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:39.953-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:39.953-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:39.953-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:43:39.953-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:43:39.953-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
CheckpointManager 2017-04-25T13:43:39.953-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
CapiUtils 2017-04-25T13:43:40.015-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:43:40.015-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:43:40.015-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:43:40.016-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
StatisticsManager 2017-04-25T13:43:40.016-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:43:40.016-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
XDCRFactory 2017-04-25T13:43:40.016-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:43:40.024-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 is constructed, start it
GenericPipeline 2017-04-25T13:43:40.025-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[186 214 269 274 338 218 234 329 184 205 242 279 179 189 224 315 215 247 178 260 305 173 210 228 334 196 217 251 220 202 227 323 221 265 267 249 298 319 270 291 337 195 263 285 304 203 283 333 303 331 245 336 174 206 284 216 223 314 257 238 259 296 320 194 199 230 243 182 302 226 308 171 229 273 324 248 246 225 288 321 191 222 289 299 307 192 266 278 281 282 335 253 290 312 275 237 264 236 252 310 197 172 198 231 325 188 261 211 256 300 313 190 193 276 254 175 207 219 241 301 295 176 212 286 318 328 185 280 327 326 277 177 232 235 292 306 244 272 287 201 250 213 311 339 187 233 262 181 200 239 294 309 297 316 240 204 209 255 180 183 271 322 330 268 293 332 340 208 258 317] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[38 45 3 82 150 37 156 164 166 29 61 107 84 110 169 104 118 63 133 151 31 43 75 87 89 137 28 66 100 147 21 55 127 70 93 4 12 59 111 7 22 65 13 26 56 146 5 98 0 33 50 57 94 161 8 131 62 90 116 52 76 132 18 51 2 14 86 148 153 154 16 113 6 88 114 158 96 125 23 71 120 155 157 15 112 134 27 48 80 97 105 160 25 108 119 40 17 91 92 130 139 11 30 74 78 64 124 149 1 73 85 159 34 95 142 152 135 41 81 117 170 136 109 162 58 60 163 165 143 72 102 9 39 53 99 20 36 47 123 128 115 138 145 35 54 168 19 77 79 121 126 144 24 122 167 32 83 103 140 42 44 46 106 141 49 68 129 10 67 69 101]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[6 88 114 158 96 125 23 71 120 155 157 15 112 134 27 48 80 97 105 160 25 108 119 40 17 91 92 130 139 11 30 74 78 64 124 149 1 73 85 159 34 95 142 152 135 41 81 117 170 136 109 162 58 60 163 165 143 72 102 9 39 53 99 20 36 47 123 128 115 138 145 35 54 168 19 77 79 121 126 144 24 122 167 32 83 103 140 42 44 46 106 141 49 68 129 10 67 69 101 38 45 3 82 150 37 156 164 166 29 61 107 84 110 169 104 118 63 133 151 31 43 75 87 89 137 28 66 100 147 21 55 127 70 93 4 12 59 111 7 22 65 13 26 56 146 5 98 0 33 50 57 94 161 8 131 62 90 116 52 76 132 18 51 2 14 86 148 153 154 16 113] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[191 222 289 299 307 192 266 278 281 282 335 253 290 312 275 237 264 236 252 310 197 172 198 231 325 188 261 211 256 300 313 190 193 276 254 175 207 219 241 301 295 176 212 286 318 328 185 280 327 326 277 177 232 235 292 306 244 272 287 201 250 213 311 339 187 233 262 181 200 239 294 309 297 316 240 204 209 255 180 183 271 322 330 268 293 332 340 208 258 317 186 214 269 274 338 218 234 329 184 205 242 279 179 189 224 315 215 247 178 260 305 173 210 228 334 196 217 251 220 202 227 323 221 265 267 249 298 319 270 291 337 195 263 285 304 203 283 333 303 331 245 336 174 206 284 216 223 314 257 238 259 296 320 194 199 230 243 182 302 226 308 171 229 273 324 248 246 225 288 321]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 filter_expression: checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 replication_type:capi active:true target_nozzle_per_node:2 log_level:Info stats_interval:1000]
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:43:40.044-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:43:40.047-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CheckpointManager 2017-04-25T13:43:40.047-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265...
GenericSupervisor 2017-04-25T13:43:40.047-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:43:40.047-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:43:40.047-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
GenericSupervisor 2017-04-25T13:43:40.047-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:43:40.055-05:00 [INFO] Checkpointing starts in 1801.89 sec
PipelineRuntimeCtx 2017-04-25T13:43:40.056-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:43:40.056-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:43:40.064-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:43:40.130-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:43:40.135-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:43:40.139-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:43:40.289-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:43:40.406-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:43:41.197-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:43:41.156-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:43:42.351-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:43:42.354-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:43:42.358-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
StatisticsManager 2017-04-25T13:43:45.396-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:43:45.360-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:43:45.756-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:43:45.667-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
GenericSupervisor 2017-04-25T13:43:53.874-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:43:53.890-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:43:53.896-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:43:53.909-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:43:53.920-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:43:53.921-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:43:53.921-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:43:53.921-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
ToplogyChangeDetector 2017-04-25T13:43:53.943-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:43:54.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:43:54.185-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:43:54.233-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:43:22.861335013-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:43:54.242-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:43:54.242-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:43:54.411-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:43:54.467-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:43:54.467-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:43:54.467-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:43:54.483-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
ReplicationManager 2017-04-25T13:43:54.508-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:43:54.902-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:43:56.139-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:43:56.151-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:43:56.429-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:43:56.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:43:56.568-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:43:22.861335013-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:43:56.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:43:56.958-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:43:56.958-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:43:57.020-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:43:57.021-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:43:57.021-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:43:57.191-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:43:57.191-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:43:57.209-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:43:57.230-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:43:57.230-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
2017/04/25 13:43:57 UPR_STREAMREQ for vb 100 successful
StatisticsManager 2017-04-25T13:43:57.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:43:57.591-05:00 [INFO] Send batch count=162
CapiNozzle 2017-04-25T13:43:57.744-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:57.744-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:57.770-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:43:57.982-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:58.017-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
StatisticsManager 2017-04-25T13:43:58.196-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:43:58.210-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:43:58.210-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:43:58.210-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:43:58.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:43:58.221-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:43:58.221-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:43:58.222-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 is started----------
PipelineManager 2017-04-25T13:43:58.222-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:43:58.222-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:43:58.242-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
PipelineManager 2017-04-25T13:43:58.245-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:43:58 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:43:58.288-05:00 [INFO] Send batch count=33
CapiNozzle 2017-04-25T13:43:58.324-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:58.324-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:58.342-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:43:58.364-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:43:58 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:43:58.377-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:58.386-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:43:58.411-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:43:58 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 7 successful
CapiNozzle 2017-04-25T13:43:58.485-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:58.522-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:43:58 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 227 successful
CapiNozzle 2017-04-25T13:43:58.655-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:43:58.655-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:43:58.655-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:43:58.655-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:43:58.655-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
CapiNozzle 2017-04-25T13:43:58.678-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:43:58.678-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:43:58.678-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:43:58.678-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:43:58.679-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:43:58.715-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:43:58.715-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:43:58.718-05:00 [INFO] err_list=[{"time":"2017-04-25T13:43:58.718536498-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:43:58.718-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:43:58.747-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:58.747-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:43:58 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:43:58 UPR_STREAMREQ for vb 146 successful
CapiNozzle 2017-04-25T13:43:58.772-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:43:58.775-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:43:58.775-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:43:58.787-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:44:04 UPR_STREAMREQ for vb 251 successful
StatisticsManager 2017-04-25T13:44:01.090-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:44:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 50 successful
GenericSupervisor 2017-04-25T13:44:04.619-05:00 [INFO] About to call function
2017/04/25 13:44:05 UPR_STREAMREQ for vb 154 successful
StatisticsManager 2017-04-25T13:44:05.343-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 {"Errors": "[{\"time\":\"2017-04-25T13:43:58.718536498-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 1, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 1, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

GenericSupervisor 2017-04-25T13:44:05.309-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:44:05.347-05:00 [INFO] Finish executing 0x681fb0
2017/04/25 13:44:05 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:44:05 UPR_STREAMREQ for vb 53 successful
CapiNozzle 2017-04-25T13:44:06.035-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:44:06.053-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:44:06 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 56 successful
CapiNozzle 2017-04-25T13:44:06.223-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:44:06 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 168 successful
CapiNozzle 2017-04-25T13:44:06.235-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:44:06 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 67 successful
CapiNozzle 2017-04-25T13:44:06.373-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:44:06.403-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:44:06 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:44:06 UPR_STREAMREQ for vb 194 successful
CapiNozzle 2017-04-25T13:44:07.623-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:44:07.623-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:44:07.623-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:44:07.623-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:07.623-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:07.627-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:07.638-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CheckpointManager 2017-04-25T13:44:07.729-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
CheckpointManager 2017-04-25T13:44:07.729-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:44:07.762-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:07.762-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:07.790-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
DcpNozzle 2017-04-25T13:44:07.829-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:44:07.829-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:44:07 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:44:07 UPR_STREAMREQ for vb 98 successful
ToplogyChangeDetector 2017-04-25T13:44:08.043-05:00 [INFO] Pipeline is no longer running, exit.
2017/04/25 13:44:08 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:44:08 UPR_STREAMREQ for vb 298 successful
CapiNozzle 2017-04-25T13:44:08.260-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:08.260-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:44:08.260-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:44:08.260-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:08.260-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:08.388-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:08.402-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:08.424-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:44:08.509-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:44:08.529-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
PipelineManager 2017-04-25T13:44:08.718-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:44:08.718-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:44:08.718-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:44:08.719-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
PipelineRuntimeCtx 2017-04-25T13:44:08.719-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:44:08.719-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:44:08.719-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:44:08.719-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:08.719048109 -0500 CDT
GenericSupervisor 2017-04-25T13:44:08.719-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:44:08.719-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:44:08.719-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:44:08.719-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:44:08.719-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:44:08.719-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:44:08.719-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:44:08.719-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:44:08.719-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:44:08.719-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 has been stopped
GenericPipeline 2017-04-25T13:44:08.719-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-926614265 is stopped
PipelineManager 2017-04-25T13:44:08.719-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:44:08.719-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:43:58.718536498-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:44:08.719-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:44:08.719-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
GenericPipeline 2017-04-25T13:44:08.719-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
GenericPipeline 2017-04-25T13:44:08.719-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
GenericPipeline 2017-04-25T13:44:08.719-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-926614265
AsyncListener 2017-04-25T13:44:08.719-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:44:08.720-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:44:08.720-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:44:08.720-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:44:08.720-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:44:08.720-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:44:08.720-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:44:08.720-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:44:08.720-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.720-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:08.720551855 -0500 CDT
DcpNozzle 2017-04-25T13:44:08.721-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:08.72079951 -0500 CDT
DcpNozzle 2017-04-25T13:44:08.721-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:44:08.721-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.721-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:08.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:08.722-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:44:08.722-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:08.722-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:44:08.724-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:44:08.724-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:44:08.724-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:08.724201579 -0500 CDT
DcpNozzle 2017-04-25T13:44:08.724-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:08.724211683 -0500 CDT
DcpNozzle 2017-04-25T13:44:08.724-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:44:08.724-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:44:08.724-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:08.724-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:44:08.724-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:08.724-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:44:08 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:44:08 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:44:08 Stream Closed for vb 0 StreamEnd simulated
PipelineManager 2017-04-25T13:44:11.484-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:43:58.718536498-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:44:11.709-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:44:11.712-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:44:11.712-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:44:11.813-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:44:11.826-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:44:11.830-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:44:11.830-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:44:11.831-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:44:11.831-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:44:11.831-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:44:11.831-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:44:11.832-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
GenericSupervisor 2017-04-25T13:44:11.834-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:44:11.834-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
GenericSupervisor 2017-04-25T13:44:11.834-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:11.834-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:11.834-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:11.834-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:11.834-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:44:11.834-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
CheckpointManager 2017-04-25T13:44:11.834-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
CapiUtils 2017-04-25T13:44:11.866-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:44:11.866-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:44:11.866-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:44:11.867-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
StatisticsManager 2017-04-25T13:44:11.867-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:44:11.868-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
XDCRFactory 2017-04-25T13:44:11.868-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:44:11.868-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 is constructed, start it
GenericPipeline 2017-04-25T13:44:11.869-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[258 272 320 190 254 216 209 297 289 184 189 314 183 319 211 260 270 309 318 197 246 256 300 307 232 313 326 304 331 264 288 337 207 283 334 188 203 242 194 215 219 275 279 333 281 303 305 187 220 267 177 234 252 298 225 226 316 325 210 217 204 259 263 182 280 301 173 238 241 306 322 328 296 282 293 176 239 278 290 310 294 186 200 218 277 195 315 172 212 321 251 329 340 240 332 291 271 274 287 185 311 335 247 302 265 178 202 330 339 192 312 237 250 262 224 327 269 268 324 206 245 208 286 244 285 223 266 179 213 229 230 198 243 248 261 174 222 233 231 196 221 228 191 308 317 235 257 292 181 214 227 171 205 284 199 249 253 323 336 236 295 175 180 338 255 299 276 273 193 201] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[33 68 140 163 100 102 165 28 35 111 136 29 108 145 155 83 110 112 162 39 45 64 154 158 135 88 90 98 104 120 26 34 157 1 59 61 123 130 152 17 166 168 3 65 32 46 77 150 131 73 103 107 146 169 2 9 6 71 91 105 106 127 129 23 70 80 97 126 4 22 42 58 142 53 5 13 25 85 87 132 36 44 156 14 148 66 78 89 160 60 82 101 8 40 144 62 69 95 11 27 52 54 57 113 119 137 159 15 21 115 147 153 121 170 48 63 76 86 93 134 19 31 47 56 41 12 37 49 51 30 75 133 139 0 7 72 92 109 18 141 55 84 117 167 38 99 114 143 20 67 24 96 124 164 10 50 161 43 122 94 116 149 79 125 81 128 138 151 16 74 118]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[70 80 97 126 127 129 23 22 42 4 142 58 53 5 25 85 87 132 13 44 156 36 148 14 78 89 160 66 82 101 60 40 144 8 27 52 54 57 62 69 95 11 119 137 159 113 21 115 147 153 15 63 76 86 93 121 170 48 134 19 47 56 31 41 12 49 51 37 30 133 75 7 72 92 109 139 0 141 18 84 117 167 55 99 114 143 38 67 20 96 24 164 124 50 161 10 122 43 116 149 94 125 79 128 138 151 81 74 118 16 68 140 163 33 102 165 100 35 111 136 28 108 145 155 29 110 112 162 83 45 64 154 158 39 90 98 104 120 135 88 34 157 26 1 61 123 59 152 130 166 168 17 65 3 46 32 150 77 131 73 107 146 169 103 9 2 71 91 105 106 6] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[296 176 239 282 293 278 290 310 186 200 218 277 294 195 315 212 321 172 251 329 240 332 340 291 185 271 274 287 311 247 302 335 265 178 202 330 339 192 312 237 262 250 327 224 269 268 324 206 245 208 286 244 223 266 285 179 213 229 230 198 243 174 222 233 248 261 231 191 196 221 228 317 308 181 214 227 235 257 292 171 205 284 199 249 323 336 253 236 295 175 180 338 255 299 276 273 193 201 190 258 272 320 254 209 216 289 297 183 184 189 314 319 260 211 270 309 318 246 256 300 307 197 232 313 326 304 331 264 288 337 207 283 334 188 203 194 215 219 242 275 279 333 187 220 267 281 303 305 234 252 298 177 225 226 325 316 210 217 182 204 259 263 301 280 173 238 241 306 328 322]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 log_level:Info stats_interval:1000 active:true worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 replication_type:capi filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048]
CheckpointManager 2017-04-25T13:44:11.869-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739...
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:11.880-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:44:11.880-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:44:11.880-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:44:11.880-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:44:11.880-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:44:11.880-05:00 [INFO] Checkpointing starts in 1801.954 sec
PipelineRuntimeCtx 2017-04-25T13:44:11.880-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:44:11.880-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:44:11.880-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:44:11.881-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:44:11.898-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:44:11.898-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:44:11.898-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:44:11.898-05:00 [INFO] updateStats started
CapiUtils 2017-04-25T13:44:11.899-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:44:11.899-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:44:11.899-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:44:11.900-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:44:11.909-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:44:11.927-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:44:11.957-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:44:11.957-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:44:11.957-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:44:11.979-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:44:11.979-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:44:11.989-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:44:12.006-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:44:12.128-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:44:12.146-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:44:12.146-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:44:12.175-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:44:12.176-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:44:12.184-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:44:12.847-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:44:13.992-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:44:13.993-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:44:14.003-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:44:14.004-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:44:14.004-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:44:14.005-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:44:14.024-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:44:14.024-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:44:14.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:44:14.249-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:44:14 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:44:14 UPR_STREAMREQ for vb 100 successful
DcpNozzle 2017-04-25T13:44:14.735-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:44:14.955-05:00 [INFO] Dcp nozzle is started
DcpNozzle 2017-04-25T13:44:15.015-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:44:15.015-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:44:15.019-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
GenericPipeline 2017-04-25T13:44:15.066-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:44:15.066-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 is started----------
PipelineManager 2017-04-25T13:44:15.067-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:44:15.067-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:44:15.067-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:44:15 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:44:15.317-05:00 [INFO] Send batch count=207
StatisticsManager 2017-04-25T13:44:15.982-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:44:15 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:44:15 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:44:16 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:44:16.089-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:16.090-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:44:16.131-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:44:16.133-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:44:17 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 240 successful
StatisticsManager 2017-04-25T13:44:17.594-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:44:17 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 48 successful
CapiNozzle 2017-04-25T13:44:17.743-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:44:17 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 249 successful
CapiNozzle 2017-04-25T13:44:17.775-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:44:17 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:44:17 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:44:21 UPR_STREAMREQ for vb 63 successful
StatisticsManager 2017-04-25T13:44:21.308-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:44:22 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:44:22 UPR_STREAMREQ for vb 260 successful
ToplogyChangeDetector 2017-04-25T13:44:22.222-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:44:22.151-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:44:24.427-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:43:58.718536498-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 579, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 83, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 1825, "docs_rep_queue": 1618, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 1042, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 83, "set_filtered": 0, "set_received_from_dcp": 1825, "size_rep_queue": 2062135, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 70}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 83, "docs_opt_repd": 0, "docs_rep_queue": 1618, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 83, "size_rep_queue": 2062135, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 165, "max": 71, "mean": 70.9939393939394, "min": 70}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 1, "dcp_dispatch_time": {"count": 1825, "max": 520136, "mean": 579.534, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 1825, "expiry_received_from_dcp": 0, "set_received_from_dcp": 1825}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 1825 items, sent 207 items
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:1825]
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 1825 items, sent 1825 items. streams inactive: [86 69 84 66 170 94 165 80 164 79 91 71 73 77 72 74 99 168 75 76 82 87 65 68 81 85 92 166 89 93 98 78 167 90 88 169 97 70 64 96 67 83 95]
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [180 299 189 176 284 289 264 273 261 295 171 181 195 268 279 283 177 287 290 175 183 174 191 266 280 184 263 294 173 182 187 292 178 185 275 278 296 293 172 196 288 269 270 298 179 276 197 192 262 281 194 265 267 282 286 186 188 193 199 285 190 271 277 297 274 198 272 291]
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:24.430-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:24.431-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:24.431-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:24.431-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:24.431-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
2017/04/25 13:44:24 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 64 successful
CapiNozzle 2017-04-25T13:44:24.500-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:44:24.707-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:44:24 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 276 successful
StatisticsManager 2017-04-25T13:44:24.815-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:44:24.815-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:44:24.815-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:44:24.815-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:44:24.815-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:44:24.815-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:44:24 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 94 successful
StatisticsManager 2017-04-25T13:44:24.912-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:44:24.912-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 579, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 83, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 1825, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 83, "set_filtered": 0, "set_received_from_dcp": 1825, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

CheckpointManager 2017-04-25T13:44:24.913-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
CheckpointManager 2017-04-25T13:44:24.913-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:44:24 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:44:24 UPR_STREAMREQ for vb 297 successful
DcpNozzle 2017-04-25T13:44:25.041-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:44:25.072-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
GenericSupervisor 2017-04-25T13:44:25.448-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:44:25.448-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:44:25.448-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:44:25.585-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:44:25.585-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:44:25.591-05:00 [INFO] err_list=[{"time":"2017-04-25T13:44:25.591300401-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:44:25.591-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:44:25.742-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:25.743-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:25.793-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
PipelineManager 2017-04-25T13:44:26.031-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:44:25.591300401-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:44:26.320-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:44:26.329-05:00 [INFO] Send batch count=485
CapiNozzle 2017-04-25T13:44:26.344-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:44:26.396-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:26.396-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:26.412-05:00 [INFO] Retrying update docs for vb 307 for the 1th time
CapiNozzle 2017-04-25T13:44:26.612-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:44:26.622-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:44:26.803-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:44:26.813-05:00 [INFO] Retrying update docs for vb 307 for the 2th time
CapiNozzle 2017-04-25T13:44:26.939-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:44:26.939-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:44:26.939-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:44:26.939-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:26.939-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:27.039-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:27.039-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:27.050-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:44:27.226-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:44:27.237-05:00 [INFO] Retrying update docs for vb 307 for the 3th time
CapiNozzle 2017-04-25T13:44:27.775-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:44:27.809-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:44:28.187-05:00 [ERROR] batchUpdateDocs for vb 307 failed with err receieved unexpected status code, 500, from update docs request for vb 307
.
CapiNozzle 2017-04-25T13:44:28.187-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 307 after 3 retries
CapiNozzle 2017-04-25T13:44:28.187-05:00 [ERROR] Raise error condition batch update docs failed for vb 307 after 3 retries
GenericSupervisor 2017-04-25T13:44:28.187-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:28.187-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:28.349-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:30.039-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:44:28.452-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:30.088-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:44:30.145-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:44:31.572-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:44:31.636-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:44:32.177-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:44:32.182-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:44:32.182-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:32.182-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:44:32.189-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:44:32.189-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:44:32.189-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:44:32.199-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:44:32.210-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:44:32.292-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:33.226-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:33.245-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:44:33.251-05:00 [INFO] Retrying update docs for vb 7 for the 1th time
CapiNozzle 2017-04-25T13:44:33.262-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:44:33.848-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err receieved unexpected status code, 500, from update docs request for vb 7
.
CapiNozzle 2017-04-25T13:44:33.849-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:44:33.849-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:44:33.849-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:44:33.849-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:33.849-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:33.876-05:00 [INFO] Retrying update docs for vb 7 for the 2th time
CapiNozzle 2017-04-25T13:44:33.995-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:33.995-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:34.006-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:44:34.329-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err receieved unexpected status code, 500, from update docs request for vb 7
.
CapiNozzle 2017-04-25T13:44:34.345-05:00 [INFO] Retrying update docs for vb 7 for the 3th time
CapiNozzle 2017-04-25T13:44:34.534-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:44:34.581-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
CapiNozzle 2017-04-25T13:44:38.326-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err receieved unexpected status code, 500, from update docs request for vb 7
.
CapiNozzle 2017-04-25T13:44:38.327-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 7 after 3 retries
CapiNozzle 2017-04-25T13:44:38.327-05:00 [ERROR] Raise error condition batch update docs failed for vb 7 after 3 retries
GenericSupervisor 2017-04-25T13:44:38.327-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:38.327-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:44:38.341-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:44:38.341-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:44:38.341-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:44:38.341-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
PipelineRuntimeCtx 2017-04-25T13:44:38.341-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:44:38.341-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:44:38.342-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:44:38.342-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:38.341336475 -0500 CDT
GenericSupervisor 2017-04-25T13:44:38.358-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:44:38.358-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:44:38.358-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:44:38.358-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:44:38.358-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:44:38.358-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:44:38.358-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:44:38.358-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:44:38.358-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:44:38.358-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 has been stopped
GenericPipeline 2017-04-25T13:44:38.358-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-831280739 is stopped
PipelineManager 2017-04-25T13:44:38.358-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:44:38.358-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:44:25.591300401-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:44:38.359-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:44:38.359-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
GenericPipeline 2017-04-25T13:44:38.359-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
GenericPipeline 2017-04-25T13:44:38.359-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
GenericPipeline 2017-04-25T13:44:38.360-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-831280739
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:38.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:44:38.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:38.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:38.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:44:38.363-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:44:38.363-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:44:38.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:44:38.363-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:44:38.363-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:44:38.363-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:44:38.363-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:44:38.363-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:44:38.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:44:38.363-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:44:38.363-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:44:38.363-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:44:38.364-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:44:38.364-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:44:38.364-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:44:38.364-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:38.363761008 -0500 CDT
DcpNozzle 2017-04-25T13:44:38.364-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:44:38.364-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:38.36406546 -0500 CDT
CapiNozzle 2017-04-25T13:44:38.365-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:44:38.365-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:44:38.365-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:44:38.366-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:38.368-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.369-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:44:38.369-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.369-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:44:38.369-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:38.369-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
AsyncListener 2017-04-25T13:44:38.383-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:44:38.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:44:38.446-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:44:38.447-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:38.447160307 -0500 CDT
DcpNozzle 2017-04-25T13:44:38.447-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:44:38.447-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:38.447-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:44:38 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:44:38 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:44:38.463-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:44:38.463-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:44:38.463-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:38.463-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:44:38.463-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:44:38.463-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:44:38.463872696 -0500 CDT
DcpNozzle 2017-04-25T13:44:38.463-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:44:38.463-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:44:38.464-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:44:38.991-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:44:39.190-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:44:39.219-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
XDCRFactory 2017-04-25T13:44:39.246-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:44:39.246-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:44:39.246-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:44:39.292-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:44:39.292-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:44:39.293-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:44:39.293-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:44:39.293-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:44:39.293-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:44:39.293-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:44:39.293-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:44:39.294-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
GenericSupervisor 2017-04-25T13:44:39.295-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:44:39.295-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
GenericSupervisor 2017-04-25T13:44:39.295-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:39.295-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:39.296-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:39.296-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:44:39.296-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:44:39.296-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
CheckpointManager 2017-04-25T13:44:39.296-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
CapiUtils 2017-04-25T13:44:39.327-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:44:39.327-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:44:39.327-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:44:39.328-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
StatisticsManager 2017-04-25T13:44:39.329-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:44:39.329-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
XDCRFactory 2017-04-25T13:44:39.329-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:44:39.329-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 is constructed, start it
GenericPipeline 2017-04-25T13:44:39.331-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[262 273 184 195 302 332 204 320 212 288 188 191 187 259 284 218 281 222 252 237 275 328 194 238 272 295 171 217 326 223 236 266 270 211 233 274 293 305 201 234 224 254 291 226 276 285 330 333 185 228 263 289 338 311 216 221 220 339 206 322 269 309 202 178 304 337 198 300 174 239 255 283 277 299 308 296 180 256 340 175 192 208 335 214 319 172 279 294 213 241 286 314 321 331 334 227 246 306 327 324 240 245 257 250 203 282 316 205 325 173 248 287 313 190 261 290 199 247 315 177 196 215 193 243 307 189 225 271 182 317 323 179 312 318 251 209 231 264 210 253 267 292 298 176 230 265 268 336 232 244 181 197 310 329 186 303 183 207 249 301 200 229 258 297 235 242 278 280 219 260] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[19 35 68 80 18 97 142 36 44 89 140 60 146 155 45 78 53 65 98 157 20 56 109 128 116 139 25 122 124 161 42 73 92 133 153 167 62 105 136 165 23 90 55 66 95 129 138 143 49 87 106 121 164 11 59 75 39 61 43 154 12 37 84 152 5 112 170 50 79 103 149 118 135 3 8 13 70 115 132 29 114 125 141 9 107 69 81 110 34 104 145 147 48 156 40 100 131 30 52 54 137 26 101 111 17 102 151 123 7 41 119 91 144 159 32 117 166 16 21 24 77 113 134 64 126 169 2 33 120 168 63 108 158 14 47 57 72 94 127 4 27 51 67 76 99 0 6 38 71 86 148 162 46 58 74 15 31 150 163 83 93 82 160 88 22 28 85 1 10 96 130]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[17 102 151 123 7 41 119 91 144 159 32 117 166 16 21 24 77 113 134 64 126 169 2 33 120 168 63 108 158 14 47 57 72 94 127 4 27 51 67 76 99 0 6 38 71 86 148 162 46 58 74 15 31 150 163 83 93 82 160 88 22 28 85 1 10 96 130 19 35 68 80 18 97 142 36 44 89 140 60 146 155 45 78 53 65 98 157 20 56 109 128 116 139 25 122 124 161 42 73 92 133 153 167 62 105 136 165 23 90 55 66 95 129 138 143 49 87 106 121 164 11 59 75 39 61 43 154 12 37 84 152 5 112 170 50 79 103 149 118 3 8 13 70 115 132 135 29 114 125 141 9 107 69 81 110 34 104 145 147 48 156 40 100 131 30 52 54 137 26 101 111] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[314 321 227 246 306 327 331 334 240 245 257 324 250 203 282 316 205 325 173 248 287 190 261 290 313 199 247 315 177 196 215 193 243 307 189 225 271 182 317 323 179 312 318 251 209 231 264 210 253 267 176 230 265 268 292 298 336 181 197 232 244 310 329 186 303 183 207 249 301 200 229 258 297 235 242 278 219 260 280 184 195 262 273 302 332 204 320 188 191 212 288 187 259 284 218 281 222 252 237 275 328 194 238 272 295 171 217 326 223 236 266 270 211 233 274 293 305 201 234 224 254 291 226 276 285 185 228 263 289 330 333 338 216 221 311 220 339 206 269 309 322 202 178 304 198 300 337 174 239 255 283 277 299 308 180 256 296 175 192 208 335 340 214 319 172 213 241 279 294 286]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: worker_batch_size:500 target_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:44:39.331-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130...
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:44:39.366-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:44:39.366-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:44:39.366-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:44:39.367-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:44:39.367-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:44:39.367-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:44:39.367-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:44:39.368-05:00 [INFO] Checkpointing starts in 1800.258 sec
PipelineRuntimeCtx 2017-04-25T13:44:39.368-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:44:39.368-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:44:39.374-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:44:39.374-05:00 [INFO] checkpointing rountine started
ToplogyChangeDetector 2017-04-25T13:44:39.397-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:44:39.399-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:44:39.399-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:44:39.399-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:44:39.402-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:44:39.402-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:44:39.412-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:44:39.527-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:44:39.546-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:44:39.633-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:44:39.812-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:44:39.949-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:44:40.342-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:44:40.529-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:44:40.551-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:44:40.612-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:44:40.612-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:44:40.612-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
StatisticsManager 2017-04-25T13:44:40.634-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:44:40.656-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:44:40.656-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:44:40.665-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:44:40.712-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:44:40.712-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:44:40.712-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
PipelineManager 2017-04-25T13:44:40.783-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:44:25.591300401-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:44:40.785-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:44:40.785-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:44:41.751-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:44:41.766-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:44:41.766-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
CheckpointManager 2017-04-25T13:44:41.694-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:44:41.731-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:44:41.831-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:44:41.831-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:44:41.832-05:00 [INFO] All parts has been started
CheckpointManager 2017-04-25T13:44:41.651-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:44:41.684-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
StatisticsManager 2017-04-25T13:44:42.344-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:44:42.407-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:44:42.749-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
GenericPipeline 2017-04-25T13:44:42.993-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 is started----------
PipelineManager 2017-04-25T13:44:42.993-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:44:42.993-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:44:42.997-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:44:43 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 121 successful
StatisticsManager 2017-04-25T13:44:43.709-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:44:43 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:44:43 UPR_STREAMREQ for vb 21 successful
ReplicationManager 2017-04-25T13:44:44.410-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:44:44.413-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:44:44 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 25 successful
StatisticsManager 2017-04-25T13:44:44.597-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:44:44 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:44:44 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:44:45 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:44:45 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:44:45 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:44:45 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:44:45 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:44:45 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:44:46 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:44:46 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:44:46 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:44:46 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:44:47 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:44:48 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:44:48 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 67 successful
GenericSupervisor 2017-04-25T13:44:50.004-05:00 [INFO] About to call function
2017/04/25 13:44:50 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 95 successful
StatisticsManager 2017-04-25T13:44:48.892-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:44:50.008-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:44:25.591300401-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 31, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:44:50.008-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:44:50.008-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:44:50.008-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:44:50.008-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [62 70 91 76 81 160 170 163 79 84 86 65 166 159 77 90 95 67 85 88 89 75 93 97 96 66 74 68 98 87 72 94 167 169 82 73 92 69 168 78 71 83 80 99]
StatisticsManager 2017-04-25T13:44:50.008-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [267 174 197 273 284 289 269 296 181 192 283 274 171 186 293 198 278 297 275 295 298 188 190 191 173 180 189 281 187 193 299 282 175 184 285 182 199 194 195 288 279 294 183 185 178 290 179 272 292 277 280 172 176 177 286 196 287 291 276]
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:44:50.009-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
2017/04/25 13:44:50 UPR_STREAMREQ for vb 267 successful
CheckpointManager 2017-04-25T13:44:50.344-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
CheckpointManager 2017-04-25T13:44:50.344-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:44:50.507-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:44:50.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:44:50 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:44:50 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:44:50.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:44:51.169-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:44:51.083-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:44:51.153-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:44:51.193-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:44:51.193-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:44:51.486-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:44:53.164-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:44:53.623-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:44:54.644-05:00 [INFO] Send batch count=196
StatisticsManager 2017-04-25T13:44:54.654-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:44:54.727-05:00 [INFO] Send batch count=120
CapiNozzle 2017-04-25T13:44:54.818-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:54.818-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:54.831-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:44:55.088-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:55.103-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
DcpNozzle 2017-04-25T13:44:55.173-05:00 [INFO] Incrementing dcp miss count for dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0. Dcp miss count = 1
PipelineManager 2017-04-25T13:44:55.260-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:44:55.341-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:55.368-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:44:55.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:44:55.695-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:55.695-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:44:55.695-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:44:55.695-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:44:55.695-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:44:55.705-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:44:55.705-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:44:55.707-05:00 [INFO] err_list=[{"time":"2017-04-25T13:44:55.707880399-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:44:55.708-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:44:55.782-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:55.782-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:55.793-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:44:56.044-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:56.060-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:44:56.299-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:56.310-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:44:56.402-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:44:56.402-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 {"Errors": "[{\"time\":\"2017-04-25T13:44:55.707880399-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 128, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 8287, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 128, "set_filtered": 0, "set_received_from_dcp": 8287, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:44:56.534-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:44:56.534-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:44:56.535-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:44:56.535-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:56.535-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:56.608-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:56.608-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:56.619-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:44:56.912-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:44:56.923-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:44:57.275-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:44:57.288-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:44:57.620-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:44:57.620-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:44:57.620-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:44:57.620-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:44:57.620-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:44:57.694-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:44:57.694-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:44:57.710-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:44:58.138-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:44:58.155-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:44:58.830-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
GenericSupervisor 2017-04-25T13:44:59.546-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:44:59.547-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:44:59.576-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
GenericSupervisor 2017-04-25T13:44:59.577-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:45:00.706-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:45:00.791-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:45:00.792-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:45:00.792-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:45:00.792-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:45:00.792-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:45:00.877-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:45:00.877-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:45:00.892-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:45:01.337-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:45:01.352-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:45:01.807-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:45:01.825-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:45:02.465-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:45:02.465-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:45:02.465-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:45:02.465-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:45:02.465-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:45:02.577-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:45:02.578-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:45:02.602-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:45:44.832-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:45:44.832-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:45:44.833-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:45:44.850-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:45:44.850-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:45:44.850-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:45:44.850-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
PipelineRuntimeCtx 2017-04-25T13:45:44.850-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:45:44.850-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:45:44.850-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:45:44.850-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:45:44.850-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:45:44.870-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:45:44.850805929 -0500 CDT
GenericSupervisor 2017-04-25T13:45:44.892-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:45:44.893-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:45:44.893-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:45:44.894-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:45:44.894-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:45:44.894-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:45:44.894-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:45:44.897-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 has been stopped
GenericPipeline 2017-04-25T13:45:44.900-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-293435130 is stopped
PipelineManager 2017-04-25T13:45:44.900-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:45:44.902-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:44:55.707880399-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:45:44.903-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:45:44.906-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
GenericPipeline 2017-04-25T13:45:44.906-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
GenericPipeline 2017-04-25T13:45:44.906-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
GenericPipeline 2017-04-25T13:45:44.910-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-293435130
AsyncListener 2017-04-25T13:45:44.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:45:44.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:45:44.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:45:44.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:45:44.910-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:45:44.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:45:44.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:45:44.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:45:44.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:45:44.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:45:44.913-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:45:44.914-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:45:44.914-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:45:44.914-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:45:44.914-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:45:44.914-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:45:44.914-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:45:44.914-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
panic: send on closed channel

goroutine 3656 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc3c30de240, 0x8edb60, 0xc212f233b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc3c324f0e0, 0x8ed920, 0xc2907afd90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3c2fe2d80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 4 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 4 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc2080dc060, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080dc060, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080dc000, 0xc2080e7000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc20802b910)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080d2010, 0xc2080e7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc2080e8000)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc2080e8000, 0xc2080ee001, 0x5ff, 0x5ff, 0xc20800bc40, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc2080d40e0, 0xc2080ee001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080ea000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080ea000, 0x8f07a0, 0xc20806e440, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc20806e420, 0xc20801fb80, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bb40, 0x7fed17280cf8, 0xc20806e420, 0x0, 0x0, 0xc20801fb80, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bb40, 0x7fed17280cf8, 0xc20806e420, 0xc208086280, 0xc20801f580, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bb40, 0x7fed17280cf8, 0xc20806e420)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080c0480, 0xc2080bbf90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080bbf90, 0xc2080c0480, 0x7fed1727f480, 0xc2080d4020, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080c0480, 0xc2080c04b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 18 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc20807e0b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 19 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc20807e0b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 13 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc2080dc290, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080dc290, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080dc230, 0xc208108000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc20812e268)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080d2068, 0xc208108000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc2080d2068, 0xc20807e108, 0xc208108000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080d43c0, 0xc208108000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e83c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080e83c0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080e83c0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080d87b0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080d87b0, 0xc20816c000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc20800b700, 0xc20816c000, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc20800b700, 0xc20816c000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc20800b780, 0xc20816c000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc208168000, 0xc2080daa00, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc208168000, 0x80f160, 0xc20800ba40, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 3592 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3252aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 23 [syscall, 4 minutes]:
syscall.Syscall(0x0, 0x0, 0xc2080c5000, 0x1000, 0xc20803b270, 0xc2081040a0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc2080c5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc2080c5000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc2080c5000, 0x1000, 0x1000, 0xc20803b270, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc2080c5000, 0x1000, 0x1000, 0xc2080de3c0, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc2080e8360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc2080e8360, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 24 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc2080842a0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 25 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080842b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 26 [select]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 27 [select, 4 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 28 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2081225a0, 0x11, 0xc208093fc0, 0xc20806e780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2081225a0, 0x11, 0xc208093fc0, 0xc20806e780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2081225a0, 0x11, 0xc208093fc0, 0xc20806e780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc20814c910)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 29 [select, 4 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20802b980, 0xf, 0xc208092fc0, 0xc20806e780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20802b980, 0xf, 0xc208092fc0, 0xc20806e780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20802b980, 0xf, 0xc208092fc0, 0xc20806e780, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc20814caa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 30 [select]:
net/http.(*persistConn).roundTrip(0xc3960fc790, 0xc2908539e0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc3eaa16750, 0xc3eacb4840, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3eaa16750, 0x7fed1727f0d0, 0xe94600, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3eaa16750, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3eaa16750, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3eaa16750, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3eaa16750, 0x7fed17282aa8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3eac69ab0, 0x9bbca0, 0x6, 0x7fed17282aa8, 0xc3eacb4800, 0x80f820, 0xc31cf4ccd8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc31cf4ccc0, 0x9bbca0, 0x6, 0x80f820, 0xc31cf4ccd8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc3eacb47e0, 0x15, 0x7fed17282aa8, 0xc3eacb4800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/goxdcr/utils.LocalPool(0xc290853990, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:85 +0x263
github.com/couchbase/goxdcr/utils.LocalBucketUUID(0xc290853990, 0xe, 0xc20802ac78, 0x8, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:370 +0x73
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080dc0e0, 0xc3e5084200, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:464 +0x12d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080dc0e0, 0xc3e5084200)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc208010700)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 31 [chan send]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc20806eb40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:110 +0x737
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 38 [IO wait, 4 minutes]:
net.(*pollDesc).Wait(0xc208010d10, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010d10, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010cb0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc2080d75d8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038140, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc208038140, 0xc208080058, 0xc20800f000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208122a20, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20806f0e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc20806f0e0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc20806f0e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080c1830)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080c1830, 0xc2080ef200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080db180, 0xc2080ef200, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080db180, 0xc2080ef200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080db1c0, 0xc2080ef200, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080ea300, 0xc2080dab80, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080ea300, 0x80f160, 0xc2080db280, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 33 [select]:
net/http.(*persistConn).roundTrip(0xc20807e2c0, 0xc2137854a0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc208052090, 0xc3eab77380, 0xc211d09fb0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3eab77380, 0x7fed1727f0d0, 0xc208052090, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc3eab77380, 0x21, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc3eab77380, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc3eab77380, 0xeb0d80, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/goxdcr/utils.doRestCall(0xc3eab77380, 0x0, 0x818220, 0xc3e4f42320, 0xeb0d80, 0xc20802b450, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:233 +0x95
github.com/couchbase/goxdcr/utils.QueryRestApiWithAuth(0xc22f4b3d80, 0x15, 0x9e0430, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:173 +0x201
github.com/couchbase/goxdcr/utils.QueryRestApi(0xc22f4b3d80, 0x15, 0x9e0430, 0xc, 0xc340e8b600, 0x9bf4c0, 0x3, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/couchbase_rest_utils.go:134 +0x17a
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostInfo(0xc20801f440, 0xc211d09260, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:113 +0x139
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostAddr(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:157 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).getHostName(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:177 +0x4c
github.com/couchbase/goxdcr/service_impl.(*XDCRTopologySvc).MyHost(0xc20801f440, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/xdcr_comp_topology_service.go:50 +0x4c
github.com/couchbase/goxdcr/replication_manager.GetReplicationInfos(0x0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:507 +0x304
github.com/couchbase/goxdcr/replication_manager.(*Adminport).doGetAllReplicationInfosRequest(0xc20806eb40, 0xc3eab76680, 0xc211d08180, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:361 +0x73
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc20806eb40, 0xc3eab76680, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:188 +0x7d9
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc20806eb40, 0xc215da2020, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc215da2020, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc20806eb58)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 34 [IO wait]:
net.(*pollDesc).Wait(0xc208010c30, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010c30, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208010bd0, 0x0, 0x7fed1727eed8, 0xc343821078)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038138, 0xc20802ce38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc208038138, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc20806ef00, 0x7fed172827a0, 0xc208038138, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 35 [select, 4 minutes]:
net/http.(*persistConn).readLoop(0xc208080000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 36 [select, 4 minutes]:
net/http.(*persistConn).writeLoop(0xc208080000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 57 [IO wait]:
net.(*pollDesc).Wait(0xc2080dcae0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080dcae0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080dca80, 0xc2081c0000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc3c77dba50)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080d2108, 0xc2081c0000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc2080d2108, 0xc20807e268, 0xc2081c0000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081be420, 0xc2081c0000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e8c60)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e8c60, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807e210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4125 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734c20, 0x1dcd6500, 0xc3c3221400, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c32537c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c32537c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 58 [select]:
net/http.(*persistConn).writeLoop(0xc20807e210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1424 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x2, 0xc2a2153400, 0xea, 0x100, 0xc2ca9a0000, 0xcf, 0x100, 0xc2648e50e0, 0xc2648e5020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 68 [IO wait]:
net.(*pollDesc).Wait(0xc2080dcd10, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080dcd10, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080dccb0, 0xc20812a000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc3ce382288)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080d2190, 0xc20812a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc2080d2190, 0xc2080809f8, 0xc20812a000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208115820, 0xc20812a000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e9560)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e9560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080809a0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2325 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0x12e, 0xc2a0e51d00, 0xec, 0x100, 0xc320fa1800, 0xb3, 0x100, 0xc30119d200, 0xc30119d140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 69 [select]:
net/http.(*persistConn).writeLoop(0xc2080809a0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1428 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc9200, 0xcc, 0xc2b2bed300, 0xec, 0x100, 0xc2ce6da800, 0xb7, 0x100, 0xc2662ac900, 0xc2662ac840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4119 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc22f388100, 0x3b9aca00, 0xc3c32211d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3c30b8b00, 0x7fed17288870, 0xc3c30de480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fed17288870, 0xc3c30de480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 379 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208065440, 0xca, 0xc2285c8800, 0xec, 0x100, 0xc22b619800, 0xbc, 0x100, 0xc210f87920, 0xc210f87860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4093 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc20806eea0, 0x7fed17282850, 0xc33eb3c500, 0xc3eab76820)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038118, 0x7fed17282850, 0xc33eb3c500, 0xc3eab76820)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fed17282850, 0xc33eb3c500, 0xc3eab76820)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc20806ef00, 0x7fed17282850, 0xc33eb3c500, 0xc3eab76820)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc33eb3c280)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 4091 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc20806eea0, 0x7fed17282850, 0xc33eb3c3c0, 0xc3eab76680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038118, 0x7fed17282850, 0xc33eb3c3c0, 0xc3eab76680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fed17282850, 0xc33eb3c3c0, 0xc3eab76680)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc20806ef00, 0x7fed17282850, 0xc33eb3c3c0, 0xc3eab76680)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc33eb3c140)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 1038 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f18c0, 0x12d, 0xc27a4a7b00, 0xec, 0x100, 0xc260328000, 0xcf, 0x100, 0xc24ff17680, 0xc24ff175c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2097 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba480, 0x2, 0xc32a08b800, 0xea, 0x100, 0xc33cee1800, 0xcf, 0x100, 0xc2faf05080, 0xc2faf23e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 878 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x0, 0xc27a4a7700, 0xea, 0x100, 0xc27a50e600, 0x3f, 0x40, 0xc27a648060, 0xc27a50df80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3624 [runnable]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3c7830600, 0xc3c77fa300, 0xc3c77fa360)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 2379 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x3, 0xc26e7ca600, 0xea, 0x100, 0xc357f0a800, 0x93, 0x100, 0xc3584b0f00, 0xc3584b0e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 973 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x64, 0xc2800af300, 0xec, 0x100, 0xc282032800, 0xc9, 0x100, 0xc2838c5b00, 0xc2838c5a40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4113 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc20806eea0, 0x7fed17282850, 0xc3e4f48140, 0xc3eaa16820)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038118, 0x7fed17282850, 0xc3e4f48140, 0xc3eaa16820)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fed17282850, 0xc3e4f48140, 0xc3eaa16820)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc20806ef00, 0x7fed17282850, 0xc3e4f48140, 0xc3eaa16820)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3e4f48000)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2438 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0xc9, 0xc366158800, 0xec, 0x100, 0xc365ead000, 0xbd, 0x100, 0xc3662927e0, 0xc366292720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1006 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x0, 0xc2901c8400, 0xea, 0x100, 0xc2902a8000, 0x9f, 0x100, 0xc290283680, 0xc2902835c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1266 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc9200, 0xc8, 0xc2aba4f000, 0xec, 0x100, 0xc2a2380800, 0xbb, 0x100, 0xc2abb5bf80, 0xc2abb5bec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1398 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x64, 0xc2c4e77300, 0xec, 0x100, 0xc2d6a92800, 0xc9, 0x100, 0xc2d09ed860, 0xc2d09ed7a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4136 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f320)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 321 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208064480, 0x3, 0xc221c0c400, 0xea, 0x100, 0xc220f3a800, 0xb5, 0x100, 0xc221f0ea80, 0xc221f0e9c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2667 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc366273d40, 0x0, 0xc383664b00, 0xea, 0x100, 0xc38315d000, 0xa2, 0x100, 0xc3836614a0, 0xc3836613e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 353 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208065440, 0xc8, 0xc221c0d400, 0xec, 0x100, 0xc22be5f800, 0xbb, 0x100, 0xc22c806420, 0xc22c806360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 343 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208065440, 0xc9, 0xc2115ab500, 0xec, 0x100, 0xc2216e5000, 0xbd, 0x100, 0xc21b2c04e0, 0xc21b2c0420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3590 [runnable]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253220)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 324 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208065440, 0x12d, 0xc20805de00, 0xec, 0x100, 0xc211c19800, 0xcf, 0x100, 0xc215bf75c0, 0xc215bf7500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4131 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734da0, 0x1dcd6500, 0xc3c3221470, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253b30, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253b30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1257 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x0, 0xc2ab8e8c00, 0xea, 0x100, 0xc2ab878600, 0x3f, 0x40, 0xc2ab8a7bc0, 0xc2ab8a7b00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1696 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a386900, 0xc9, 0xc2e5533a00, 0xec, 0x100, 0xc2e7539000, 0xbd, 0x100, 0xc2e7eca6c0, 0xc2e7eca600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 553 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0x64, 0xc24124ce00, 0xec, 0x100, 0xc2123d4000, 0x3a, 0x40, 0xc238e40d20, 0xc238e40c60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3593 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253400)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 415 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208064480, 0x64, 0xc208170f00, 0xec, 0x100, 0xc23b48a000, 0xc9, 0x100, 0xc23b5792c0, 0xc23b579200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2355 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x64, 0xc27132d200, 0xec, 0x100, 0xc30be2e800, 0xc9, 0x100, 0xc310a4f1a0, 0xc310a4f0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4122 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734b60, 0x1dcd6500, 0xc3c3221480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253d10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253d10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 684 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38d80, 0xd0, 0xc24a32a600, 0xec, 0x100, 0xc24acae800, 0xb2, 0x100, 0xc25e514120, 0xc25e514060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 116 [select]:
net/http.(*persistConn).writeLoop(0xc20807e370)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 115 [IO wait]:
net.(*pollDesc).Wait(0xc2080dd5d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080dd5d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080dd570, 0xc20acfc000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc3c7892d18)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080d2160, 0xc20acfc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc2080d2160, 0xc20807e3c8, 0xc20acfc000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2083748a0, 0xc20acfc000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20836cd20)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20836cd20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807e370)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 656 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38d80, 0xc9, 0xc2115aa000, 0xec, 0x100, 0xc25d76e800, 0xaf, 0x100, 0xc222cc5d40, 0xc222cc5c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2303 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x4, 0xc2a156c000, 0xea, 0x100, 0xc32d4d2000, 0xa2, 0x100, 0xc28b911800, 0xc28b911740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1064 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x7, 0xc2285c8e00, 0xea, 0x100, 0xc297eff800, 0xbe, 0x100, 0xc298074c00, 0xc298074b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4110 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0xcc, 0xc3e7371f00, 0xec, 0x100, 0xc3e9911000, 0xb7, 0x100, 0xc3ea4afda0, 0xc3ea4afce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2418 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0x12f, 0xc2115aa400, 0xec, 0x100, 0xc363e64800, 0xb3, 0x100, 0xc364620900, 0xc364620840, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 145 [syscall, 3 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 1101 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f18c0, 0xd0, 0xc2a0e50d00, 0xec, 0x100, 0xc2a1138000, 0xb2, 0x100, 0xc2a113f320, 0xc2a113f260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3587 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c32535e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 1393 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc9200, 0x12c, 0xc2cc049200, 0xec, 0x100, 0xc2988c1800, 0xc4, 0x100, 0xc2c4b8efc0, 0xc2c4b8ef00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 135 [runnable]:
net.(*pollDesc).Wait(0xc20da07e90, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20da07e90, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20da07e30, 0xc20da03000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc213785030)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080381f0, 0xc20da03000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc2080381f0, 0xc20807e318, 0xc20da03000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20da129c0, 0xc20da03000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20820b7a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20820b7a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807e2c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 136 [select]:
net/http.(*persistConn).writeLoop(0xc20807e2c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2117 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba480, 0x1, 0xc2ffc78700, 0xea, 0x100, 0xc33b70d800, 0xc7, 0x100, 0xc2f7a984e0, 0xc2f7a98420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1366 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc9200, 0xca, 0xc2c4e76200, 0xec, 0x100, 0xc2c4388800, 0xbc, 0x100, 0xc2c52bf380, 0xc2c52bf2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3685 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3c2fe3080, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:367 +0x2392
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 4140 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f620)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2391 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x2, 0xc2285c8b00, 0xea, 0x100, 0xc362409800, 0xcf, 0x100, 0xc362cd9680, 0xc362cd95c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1078 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f18c0, 0xce, 0xc24a32ad00, 0xec, 0x100, 0xc296e65800, 0xab, 0x100, 0xc297481260, 0xc2974811a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1602 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x0, 0xc27a4a6100, 0xea, 0x100, 0xc2c25c7800, 0x44, 0x80, 0xc24c35b920, 0xc24c35b860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 662 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0x64, 0xc24e576c00, 0xec, 0x100, 0xc254a6f800, 0xac, 0x100, 0xc2551e9b00, 0xc2551e9a40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1022 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f18c0, 0xca, 0xc27e736200, 0xec, 0x100, 0xc25c44a000, 0xbc, 0x100, 0xc2664c3ec0, 0xc2664c3e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1089 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f18c0, 0x134, 0xc2115ab300, 0xec, 0x100, 0xc29f254000, 0xc6, 0x100, 0xc29f2d98c0, 0xc29f2d9800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 273 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208065440, 0x12c, 0xc212f31b00, 0xec, 0x100, 0xc216269800, 0x9d, 0x100, 0xc2165d2c00, 0xc2165d2b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 391 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208065440, 0xcd, 0xc212f31400, 0xec, 0x100, 0xc235c13800, 0xba, 0x100, 0xc2369d64e0, 0xc2369d6420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1759 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x1, 0xc3102b8800, 0xea, 0x100, 0xc30e694800, 0xc7, 0x100, 0xc310310fc0, 0xc310310f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 354 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208064480, 0x2, 0xc21877c800, 0xea, 0x100, 0xc2264f2800, 0xcf, 0x100, 0xc227157c80, 0xc227157bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1350 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x66, 0xc2ba885a00, 0xec, 0x100, 0xc2c12cb000, 0xc8, 0x100, 0xc2c1a785a0, 0xc2c1a784e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 389 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208064480, 0x65, 0xc212f30400, 0xec, 0x100, 0xc230b21800, 0xc6, 0x100, 0xc230dada40, 0xc230dad980, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 640 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38d80, 0xcf, 0xc213729e00, 0xec, 0x100, 0xc249802800, 0xac, 0x100, 0xc24a0b3320, 0xc24a0b3260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4120 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734ae0, 0x1dcd6500, 0xc3c32212f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253040, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253040)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 740 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38d80, 0x12e, 0xc26e7cb800, 0xec, 0x100, 0xc26e680800, 0xb3, 0x100, 0xc26ea46a20, 0xc26ea46960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 622 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0x0, 0xc22d6f5c00, 0xea, 0x100, 0xc21fd2d800, 0xb8, 0x100, 0xc2192a1800, 0xc2192a1740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2577 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36689e000, 0x12c, 0xc36facec00, 0xec, 0x100, 0xc36ffcec00, 0x41, 0x80, 0xc37001a240, 0xc37001a180, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3628 [sleep]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3c30de480, 0xc3d55100cc, 0xc2080b5b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:760 +0x287
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3c30de480, 0xc3cbed9420, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3c30de480, 0xc3cbed9420, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3c30de480, 0xc3c30e7b20, 0xc3c30de620, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 4132 [chan send]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3c7b32090, 0x14db0000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3c2fe2d80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3c2fe2d80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f020)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 716 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0xa, 0xc208170600, 0xeb, 0x100, 0xc26d36e000, 0xc4, 0x100, 0xc26d3caa80, 0xc26d3ca9c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3353 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b5680, 0x0, 0xc3af7d0000, 0xea, 0x100, 0xc3af5ff800, 0x7c, 0x80, 0xc3af724ea0, 0xc3af724de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1707 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x69, 0xc2e848f800, 0xec, 0x100, 0xc2e8c0f000, 0xba, 0x100, 0xc2e968c540, 0xc2e968c480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4134 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f1a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3486 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b5680, 0x64, 0xc3b7118200, 0xec, 0x100, 0xc3b6aae800, 0xc9, 0x100, 0xc3b718e2a0, 0xc3b718e1e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2037 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba6c0, 0xc8, 0xc32a08be00, 0xec, 0x100, 0xc32d036000, 0xbb, 0x100, 0xc32cfedc20, 0xc32cfedb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4146 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385faa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1002 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f18c0, 0xc9, 0xc28d2b7300, 0xec, 0x100, 0xc28ddba000, 0xbd, 0x100, 0xc28de2c8a0, 0xc28de2c7e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2081 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba6c0, 0x12e, 0xc3102b9600, 0xec, 0x100, 0xc33b928000, 0xb3, 0x100, 0xc2f86bee40, 0xc2f86bed80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1741 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a386900, 0x131, 0xc2fc96e400, 0xec, 0x100, 0xc2fc932000, 0xbf, 0x100, 0xc2fc9a3140, 0xc2fc9a3080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3586 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c32537c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4128 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734ce0, 0x1dcd6500, 0xc3c3221490, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253ea0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253ea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 309 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208064480, 0x0, 0xc21877c900, 0xea, 0x100, 0xc2182af800, 0xc8, 0x100, 0xc218a23ec0, 0xc218a23e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 371 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc208064480, 0x4, 0xc21877c200, 0xea, 0x100, 0xc219980800, 0xa2, 0x100, 0xc215483860, 0xc2154837a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1052 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x5, 0xc24e576a00, 0xea, 0x100, 0xc25f862800, 0xbc, 0x100, 0xc27f6cc6c0, 0xc27f6cc600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 714 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38d80, 0xcc, 0xc212f30c00, 0xec, 0x100, 0xc26c575800, 0xb7, 0x100, 0xc26cc329c0, 0xc26cc32900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2669 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36689e000, 0x12d, 0xc383664f00, 0xec, 0x100, 0xc384af5000, 0xd0, 0x100, 0xc3851f2a80, 0xc3851f29c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3501 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b5680, 0x7, 0xc3be8e4100, 0xea, 0x100, 0xc3be86a000, 0xbe, 0x100, 0xc3be926ba0, 0xc3be926ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4129 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734d20, 0x1dcd6500, 0xc3c3221270, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3252aa0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3252aa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1652 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a386900, 0x12c, 0xc212f30e00, 0xec, 0x100, 0xc2c2c40800, 0xc1, 0x100, 0xc2aa9e6fc0, 0xc2aa9e6f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2363 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0xc8, 0xc26f1afc00, 0xec, 0x100, 0xc356c03000, 0xbb, 0x100, 0xc35726bc20, 0xc35726bb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3583 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253d10)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4133 [runnable]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3c30de240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:145 +0x108
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3c30de240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f0e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1024 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x2, 0xc27e737900, 0xea, 0x100, 0xc255536000, 0xcf, 0x100, 0xc26455ac00, 0xc26455ab40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2041 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba480, 0x0, 0xc32d3f9700, 0xea, 0x100, 0xc330bee000, 0x50, 0x80, 0xc330c248a0, 0xc330c247e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2065 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba480, 0x64, 0xc21877cb00, 0xec, 0x100, 0xc27af36800, 0xc9, 0x100, 0xc26d6f85a0, 0xc26d6f84e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3591 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253ea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4092 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc20806eea0, 0x7fed17282850, 0xc33eb3c460, 0xc3eab76750)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc208038118, 0x7fed17282850, 0xc33eb3c460, 0xc3eab76750)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fed17282850, 0xc33eb3c460, 0xc3eab76750)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc20806ef00, 0x7fed17282850, 0xc33eb3c460, 0xc3eab76750)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc33eb3c1e0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2329 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x66, 0xc2800ae700, 0xec, 0x100, 0xc31edbb800, 0xc8, 0x100, 0xc2f54346c0, 0xc2f5434600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2299 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0xca, 0xc2a2152600, 0xec, 0x100, 0xc330f4f000, 0xbc, 0x100, 0xc2e39e8600, 0xc2e39e8540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1378 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x65, 0xc2cf296500, 0xec, 0x100, 0xc2d468b000, 0xc6, 0x100, 0xc2d50f76e0, 0xc2d50f7620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 672 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0x1, 0xc24124d200, 0xea, 0x100, 0xc24e16a000, 0xc7, 0x100, 0xc25c2253e0, 0xc25c225320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 744 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0x70, 0xc26f1aed00, 0xec, 0x100, 0xc26edd3000, 0x1f4, 0x200, 0xc26f397e00, 0xc26f397d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3654 [runnable]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc3c2fe2d80, 0xc3c3585470)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 4118 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc24150ffe0, 0x3b9aca00, 0xc3c32211d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3c30b8b00, 0x7fed17288b40, 0xc3c2fe3080, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fed17288b40, 0xc3c2fe3080)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 1344 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc9200, 0xc8, 0xc2ba884100, 0xec, 0x100, 0xc2ba670000, 0x5e, 0x80, 0xc2ba71d440, 0xc2ba71d380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 686 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc240d38b40, 0xc, 0xc24a32b000, 0xeb, 0x100, 0xc25f367000, 0xb8, 0x100, 0xc25fe21800, 0xc25fe21740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1415 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x67, 0xc2b572ff00, 0xec, 0x100, 0xc2cee0b000, 0xb3, 0x100, 0xc266bda120, 0xc266bda060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1934 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba6c0, 0x12c, 0xc31bb22b00, 0xec, 0x100, 0xc31c061200, 0x24, 0x40, 0xc31bf85860, 0xc31bf857a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4121 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734b20, 0x1dcd6500, 0xc3c3221300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253220, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253220)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4141 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f6e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1745 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x65, 0xc2fc96f500, 0xec, 0x100, 0xc2fdfe9800, 0xc6, 0x100, 0xc2fe778120, 0xc2fe778060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3584 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3252c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2389 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0x12d, 0xc22d6f5200, 0xec, 0x100, 0xc362408000, 0xcf, 0x100, 0xc3624baf00, 0xc3624bae40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1105 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x3, 0xc2a156ce00, 0xea, 0x100, 0xc2a15d3800, 0xb5, 0x100, 0xc2a19f74a0, 0xc2a19f73e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1661 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x64, 0xc2115aad00, 0xec, 0x100, 0xc2be09e800, 0xc9, 0x100, 0xc261874f60, 0xc261874ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2063 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba6c0, 0x12c, 0xc333ee7000, 0xec, 0x100, 0xc2ac793800, 0xaf, 0x100, 0xc223512d80, 0xc223512cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1093 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2711f1680, 0x1, 0xc20805ca00, 0xea, 0x100, 0xc29f9a0000, 0xc7, 0x100, 0xc29fa2e9c0, 0xc29fa2e900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3627 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc3c30de480, 0xc3c30e7c00, 0xc3c30de620)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 1731 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x6d, 0xc2ed243600, 0xec, 0x100, 0xc2f0e0f800, 0xb0, 0x100, 0xc2f0fe12c0, 0xc2f0fe1200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1903 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba480, 0x0, 0xc31d326700, 0xea, 0x100, 0xc31c820800, 0xb8, 0x100, 0xc31d411bc0, 0xc31d411b00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4130 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734d60, 0x1dcd6500, 0xc3c3221310, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253400, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253400)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2889 [runnable]:
net.(*pollDesc).Wait(0xc395c5f720, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc395c5f720, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc395c5f6c0, 0xc395eed000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc3d4359bc8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc390797af0, 0xc395eed000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc390797af0, 0xc3960fc7e8, 0xc395eed000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3961885e0, 0xc395eed000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc39618a540)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc39618a540, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3960fc790)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2270 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0x12f, 0xc2cf297f00, 0xeb, 0x100, 0xc2f7406600, 0x2f, 0x40, 0xc2ec4679e0, 0xc2ec467920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1720 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a386900, 0x12d, 0xc2edef0800, 0xec, 0x100, 0xc2efec5800, 0xcf, 0x100, 0xc2f044cae0, 0xc2f044ca20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3585 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3252e60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 4067 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0x12c, 0xc3d2a43700, 0xec, 0x100, 0xc3d5dcc000, 0x99, 0x100, 0xc3d5db50e0, 0xc3d5db5020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2890 [select]:
net/http.(*persistConn).writeLoop(0xc3960fc790)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4138 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f4a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1688 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a3866c0, 0x0, 0xc2cf297500, 0xea, 0x100, 0xc2e0de9000, 0x9f, 0x100, 0xc2e11e0a80, 0xc2e11e09c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3474 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b5680, 0x0, 0xc3b2af1e00, 0xea, 0x100, 0xc3b2a04000, 0xa2, 0x100, 0xc3b2b4eea0, 0xc3b2b4ede0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3594 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253b30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3683 [select]:
github.com/couchbase/goxdcr/parts.(*DcpNozzle).collectDcpDataChanLen(0xc3c2fe3080, 0xc3c7da82a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:857 +0x1ad
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:248 +0x5cf

goroutine 2268 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x3, 0xc2cf297b00, 0xea, 0x100, 0xc331b43800, 0x4d, 0x80, 0xc2ec7030e0, 0xc2ec703020, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1767 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a386900, 0xd7, 0xc30bdbc500, 0xec, 0x100, 0xc30bd36000, 0x1f4, 0x200, 0xc30bfddb60, 0xc30bfddaa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2420 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364d80, 0x65, 0xc2115aa600, 0xec, 0x100, 0xc364989800, 0xc6, 0x100, 0xc364d46060, 0xc364c8bf80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1339 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc8fc0, 0x3, 0xc2b2becb00, 0xea, 0x100, 0xc2b2f8c000, 0xb5, 0x100, 0xc2b318e600, 0xc2b318e540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1686 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc25a386900, 0xc8, 0xc2cf296200, 0xec, 0x100, 0xc270ad3000, 0xbb, 0x100, 0xc2bd5efec0, 0xc2bd5efe00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2561 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc366273d40, 0x0, 0xc36fbded00, 0xea, 0x100, 0xc36fa87000, 0x74, 0x80, 0xc36fc39e60, 0xc36fc39da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1436 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a1fc9200, 0x12e, 0xc2a0e50600, 0xec, 0x100, 0xc2c9f37800, 0xb3, 0x100, 0xc290fd01e0, 0xc290fd0120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2623 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc366273d40, 0x64, 0xc37684ed00, 0xec, 0x100, 0xc378557800, 0xc9, 0x100, 0xc378e3cc60, 0xc378e3cba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3035 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc390b18900, 0x12c, 0xc3a055b600, 0xec, 0x100, 0xc39fd36800, 0xbd, 0x100, 0xc3a0b377a0, 0xc3a0b376e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3629 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3c30de480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4047 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0x130, 0xc3d88aeb00, 0xec, 0x100, 0xc3ddb8a000, 0xbc, 0x100, 0xc3ddc23380, 0xc3ddc232c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4135 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f260)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2095 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3128ba6c0, 0xc9, 0xc32a08a200, 0xec, 0x100, 0xc33dc08000, 0xbd, 0x100, 0xc2fc14e720, 0xc2fc14e660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3097 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc390b18900, 0xcb, 0xc3a3cee100, 0xec, 0x100, 0xc3a3569800, 0xc9, 0x100, 0xc3a3d382a0, 0xc3a3d381e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3511 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b58c0, 0xce, 0xc3b756b200, 0xec, 0x100, 0xc3baf9a800, 0xab, 0x100, 0xc3bb631ec0, 0xc3bb631e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2351 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2fc364fc0, 0x130, 0xc27e737e00, 0xec, 0x100, 0xc295ad3800, 0xbc, 0x100, 0xc2ab8b61e0, 0xc2ab8b6120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3538 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b58c0, 0x132, 0xc3c092e200, 0xec, 0x100, 0xc3bfa2d000, 0xbf, 0x100, 0xc3c08d7920, 0xc3c08d7860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3093 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc390b186c0, 0x65, 0xc3a21f6800, 0xec, 0x100, 0xc3a155a800, 0xc6, 0x100, 0xc3a26bcc60, 0xc3a26bcba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2877 [IO wait]:
net.(*pollDesc).Wait(0xc395c94370, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc395c94370, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc395c94310, 0xc396302000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc396300940)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc390797ae8, 0xc396302000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc390797ae8, 0xc3960fc528, 0xc396302000, 0x1000, 0x1000, 0x7fed1726d010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc396188580, 0xc396302000, 0x1000, 0x1000, 0xc1ef23a5cc, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3962fc2a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3962fc2a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3960fc4d0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 3550 [sleep]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3c30de240, 0xc3b33d0064, 0xc340e8db50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:760 +0x287
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3c30de240, 0xc3c7781ce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3c30de240, 0xc3c7781ce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3c30de240, 0xc3c30e79d0, 0xc3c30de3e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 2659 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36689e000, 0xc8, 0xc379832700, 0xec, 0x100, 0xc379a7a000, 0xbb, 0x100, 0xc379b7a720, 0xc379b7a660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4117 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc24150ff40, 0x3b9aca00, 0xc3c32211d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3c30b8b00, 0x7fed17288870, 0xc3c30de240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fed17288870, 0xc3c30de240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 2880 [select]:
net/http.(*persistConn).writeLoop(0xc3960fc630)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2878 [select]:
net/http.(*persistConn).writeLoop(0xc3960fc4d0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4143 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f860)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2879 [IO wait]:
net.(*pollDesc).Wait(0xc395c5f560, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc395c5f560, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc395c5f500, 0xc396306000, 0x1000, 0x1000, 0x0, 0x7fed1727eed8, 0xc396300958)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc395c5a328, 0xc396306000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fed17280bf0, 0xc395c5a328, 0xc3960fc688, 0xc396306000, 0x1000, 0x1000, 0x7fed1726d010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc3962e3d00, 0xc396306000, 0x1000, 0x1000, 0xc1ef23a5cb, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3962fc300)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3962fc300, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3960fc630)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4142 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f7a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3655 [semacquire]:
sync.(*RWMutex).RLock(0xc3c7b32090)
	/usr/local/go/src/sync/rwmutex.go:36 +0x5f
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3c7b32090, 0xc3c7b80720)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:553 +0x6f1
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3684 [runnable]:
encoding/binary.bigEndian.Uint64(0xc31c45d180, 0x8, 0x507, 0x4c4)
	/usr/local/go/src/encoding/binary/binary.go:109
github.com/couchbase/gomemcached/client.makeUprEvent(0x57, 0x14b31f2d50cf0000, 0xe8115600e8, 0xc31c45d180, 0x1f, 0x507, 0xc31c45d19f, 0x24, 0x4e8, 0xc31c45d1c3, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:111 +0x158
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3c780b170, 0xc3cc419020)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:476 +0x1cc5
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 4144 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f920)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3037 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc390b186c0, 0x64, 0xc3a055bf00, 0xec, 0x100, 0xc3a0df4000, 0xa8, 0x100, 0xc3a0f581e0, 0xc3a0f58120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4139 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f560)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4037 [runnable]:
net/http.(*persistConn).roundTrip(0xc2080809a0, 0xc213734920, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc3eaa17930, 0xc24150fdc0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3eaa17930, 0x7fed1727f0d0, 0xe94600, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3eaa17930, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3eaa17930, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3eaa17930, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3eaa17930, 0x7fed17282aa8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc34381a150, 0x9bbca0, 0x6, 0x7fed17282aa8, 0xc24150fd80, 0x80f820, 0xc34385ef18, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc34385ef00, 0x9bbca0, 0x6, 0x80f820, 0xc34385ef18, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc24150fd40, 0x15, 0x7fed17282aa8, 0xc24150fd80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/goxdcr/utils.LocalPool(0xc2137348b0, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:85 +0x263
github.com/couchbase/goxdcr/utils.LocalBucket(0xc2137348b0, 0xe, 0xc20802ac78, 0x8, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:105 +0x166
github.com/couchbase/goxdcr/service_impl.(*ClusterInfoSvc).GetBucket(0xc2080380f0, 0x7fed17282f88, 0xc20801f440, 0xc20802ac78, 0x8, 0x40adf9, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/cluster_info_service.go:133 +0x25e
github.com/couchbase/goxdcr/factory.(*XDCRFactory).constructSourceNozzles(0xc208058280, 0xc3e5084880, 0xc2080da480, 0x32, 0xc2137348a0, 0x1, 0xc3d5bb4870, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:280 +0x11c
github.com/couchbase/goxdcr/factory.(*XDCRFactory).NewPipeline(0xc208058280, 0xc2080da480, 0x32, 0xc2137347a0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:94 +0x39b
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).startPipeline(0xeb0fa0, 0xc2080da480, 0x32, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:280 +0x413
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc3d4e26280, 0xc3d5aff560)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:593 +0xdc2
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc3d4e26280)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 4145 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f9e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3589 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253040)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3478 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a58b58c0, 0x133, 0xc3b54b0a00, 0xec, 0x100, 0xc3b4933800, 0xb6, 0x100, 0xc3b553b560, 0xc3b553b4a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3682 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3c2fe3080)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4123 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734ba0, 0x1dcd6500, 0xc3c3221280, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3252c80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3252c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4147 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385fb60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4072 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0x12e, 0xc3d7709c00, 0xec, 0x100, 0xc3d80a0800, 0xb3, 0x100, 0xc3d8614600, 0xc3d8614540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3551 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3c30de240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4116 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc24150fe60, 0x3b9aca00, 0xc3c32211d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3c30b8b00, 0x7fed17288b40, 0xc3c2fe2d80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fed17288b40, 0xc3c2fe2d80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3588 [select]:
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc3c3253950)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:68 +0x3e3
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 2982 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc390b186c0, 0x64, 0xc395c15000, 0xec, 0x100, 0xc39d4c6400, 0x64, 0x80, 0xc39d535680, 0xc39d5355c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4127 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734ca0, 0x1dcd6500, 0xc3c3221410, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3253950, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3253950)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3653 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3c2fe2d80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4137 [runnable]:
github.com/couchbase/goxdcr/simple_utils.func·001(0xc34385f3e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:36
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3660 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3cbeee6c0, 0xc3cc41c0c0, 0xc3cc41c120)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 4115 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xeb1a20, 0xc3eacb0990, 0xc3eacb09c0, 0xc2080103f0, 0xc3eac699d0, 0xc208122780)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:282 +0x6ed
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 4126 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734c60, 0x1dcd6500, 0xc3c32213f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c32535e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c32535e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4028 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0x12c, 0xc3cf2cf500, 0xec, 0x100, 0xc3d3189800, 0x42, 0x80, 0xc3d322e300, 0xc3d322e240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4124 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc213734be0, 0x1dcd6500, 0xc3c3221290, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc3c3252e60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7fed17288bc0, 0xc3c3252e60)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4085 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0xcb, 0xc3e06b5f00, 0xec, 0x100, 0xc3e1efc800, 0xc9, 0x100, 0xc3e2685aa0, 0xc3e26859e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3549 [select]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).selfMonitor(0xc3c30de240, 0xc3c30e7ab0, 0xc3c30de3e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:625 +0x547
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:275 +0x3bd

goroutine 4112 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3c30de480, 0xcc, 0xc3ea7a4a00, 0xec, 0x100, 0xc3e9912800, 0xb7, 0x100, 0xc3eabc6d20, 0xc3eabc6c60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31
[goport] 2017/04/25 13:45:45 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:45:46.588-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:45:46.588-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:45:46.588-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:45:46.617-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:45:46.619-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:45:46.619-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:45:46.619-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:45:46.619-05:00 [INFO] Replication manager is initialized
ReplicationManager 2017-04-25T13:45:46.619-05:00 [INFO] pollEOF: About to start stdin polling
GenericSupervisor 2017-04-25T13:45:46.619-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:45:46.619-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:45:46.619-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:45:46.619-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:45:46.619-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:45:46.619-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:45:46.619-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:45:46.619-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:45:46.619-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:45:46.619-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:45:46.619-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:45:46.619-05:00 [INFO] ReplicationManager is running
HttpServer 2017-04-25T13:45:46.619-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:45:46.620-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:45:46.620-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
ReplicationSpecChangeListener 2017-04-25T13:45:46.623-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:45:46.623-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
RemoteClusterChangeListener 2017-04-25T13:45:46.624-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:45:46.624-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
PipelineManager 2017-04-25T13:45:47.404-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:46:01.649-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:46:01.651-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:46:01.652-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:46:01.716-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:01.717-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:46:01.717-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:46:01.747-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:01.748-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:46:01.748-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:01.748-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:46:01.748-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:01.748-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:46:01.749-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:46:01.749-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:46:01.751-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
GenericSupervisor 2017-04-25T13:46:01.752-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:46:01.752-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
GenericSupervisor 2017-04-25T13:46:01.752-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:01.752-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:01.752-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:01.752-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:01.752-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:46:01.752-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
CheckpointManager 2017-04-25T13:46:01.752-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
CapiUtils 2017-04-25T13:46:01.773-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:01.773-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:01.773-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:46:01.774-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
StatisticsManager 2017-04-25T13:46:01.774-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:46:01.774-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
XDCRFactory 2017-04-25T13:46:01.774-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:46:01.774-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229 is constructed, start it
GenericPipeline 2017-04-25T13:46:01.775-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[57 167 39 64 170 127 145 72 28 83 9 79 138 29 46 67 2 14 65 146 6 42 76 106 141 5 37 95 153 158 48 51 157 13 104 149 31 140 26 36 139 12 103 150 151 60 3 63 54 165 166 74 128 24 75 163 43 99 113 35 55 116 131 81 169 23 45 123 15 22 114 10 90 121 7 86 92 107 160 33 148 59 66 70 93 100 108 73 77 58 102 118 144 62 84 136 162 109 120 88 134 122 19 41 126 137 11 96 98 119 129 133 156 8 16 97 168 49 159 20 30 71 53 17 56 68 132 142 161 94 52 18 135 27 82 101 152 61 69 80 47 85 155 44 32 34 112 164 1 91 38 105 117 124 125 89 4 78 110 111 21 25 147 154 0 40 87 115 50 130 143] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[178 207 262 271 223 298 329 243 279 322 239 337 180 229 198 212 230 288 205 278 181 193 285 287 307 319 294 326 258 340 171 333 264 184 272 304 182 188 200 232 236 252 302 228 292 183 334 202 249 297 208 227 269 305 310 175 214 244 257 274 284 172 311 189 201 220 204 273 235 231 286 296 327 339 190 266 196 282 323 246 176 299 328 179 186 206 268 300 308 218 290 336 210 217 261 295 303 270 320 324 293 216 281 224 265 225 241 263 318 331 177 187 192 338 195 222 191 291 312 209 280 306 248 276 315 316 253 255 283 259 194 233 313 335 221 251 254 256 203 260 267 317 185 245 250 301 199 213 226 242 277 289 309 174 215 219 234 240 247 325 197 275 173 237 238 314 321 211 330 332]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[113 99 55 116 131 35 169 81 45 123 23 22 114 15 90 121 10 86 92 107 160 7 148 33 66 70 93 100 59 108 77 73 102 118 144 58 84 136 162 62 120 109 134 88 122 41 126 137 19 96 98 119 129 133 156 11 16 97 168 8 159 49 30 71 20 53 56 68 132 142 161 17 94 52 135 18 82 101 152 27 69 80 61 85 155 47 44 34 112 164 32 91 1 105 117 124 125 38 89 78 110 111 4 21 147 154 25 40 87 115 0 130 143 50 167 57 64 170 39 145 127 72 28 83 79 138 9 46 67 29 14 65 146 2 42 76 106 141 6 37 95 153 158 5 51 157 48 104 149 13 140 31 36 139 26 103 150 151 12 60 63 3 165 166 54 128 74 75 163 24 43] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[172 311 189 201 220 204 273 235 231 286 296 327 339 190 266 196 282 323 246 176 299 328 179 186 206 268 308 300 218 290 336 210 217 261 295 303 270 320 324 293 216 281 224 265 225 241 263 318 331 177 187 192 338 195 222 191 291 312 209 280 306 248 276 315 316 253 255 283 259 194 233 313 335 221 251 254 256 203 260 267 317 185 245 250 301 199 213 226 242 289 309 277 174 215 219 234 240 247 325 197 275 173 237 238 314 321 211 330 332 178 207 262 271 223 298 329 243 279 322 239 337 180 229 212 230 288 198 205 278 193 181 285 287 307 319 294 326 258 340 171 333 264 184 272 304 182 188 200 232 252 302 236 228 292 183 334 202 249 297 208 227 269 305 310 175 214 244 257 274 284]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: doc_batch_size_kb:2048 source_nozzle_per_node:2 stats_interval:1000 log_level:Info replication_type:capi active:true checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:46:01.775-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229...
AsyncListener 2017-04-25T13:46:01.781-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:01.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:46:01.782-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:01.782-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:46:01.782-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:46:01.782-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:46:01.782-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:46:01.782-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:46:01.782-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:46:01.782-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:46:01.782-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:46:01.783-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:46:01.783-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:46:01.783-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:46:01.784-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:46:01.784-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:46:01.814-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:01.814-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:01.814-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:46:01.876-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:46:01.878-05:00 [INFO] validateSourceTopology...
TCPConnPoolMgr 2017-04-25T13:46:01.884-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:46:01.884-05:00 [INFO] connectionSize=5
TCPConnPoolMgr 2017-04-25T13:46:01.886-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:46:01.886-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:46:01.886-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:46:01.886-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:46:01.899-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:46:01.900-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:46:01.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:46:01.944-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:46:01.956-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:46:01.956-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:01.956-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:46:01.957-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:46:01.957-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:46:01.961-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:46:02.052-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:46:02.052-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:46:02.052-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:46:02.067-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:46:02.067-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:46:02.256-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:46:02.256-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:02.256-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:46:02.276-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:46:02.276-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:46:02.277-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:46:02.277-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229 is started----------
PipelineManager 2017-04-25T13:46:02.277-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:46:02.277-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:46:02.277-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:46:02.308-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:46:02.309-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:46:02.309-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:46:02.309-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:46:02 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 17 successful
CapiNozzle 2017-04-25T13:46:02.614-05:00 [INFO] Send batch count=44
2017/04/25 13:46:02 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 121 successful
CapiNozzle 2017-04-25T13:46:02.707-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
2017/04/25 13:46:02 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 224 successful
CapiNozzle 2017-04-25T13:46:02.729-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:46:02.769-05:00 [INFO] Send batch count=33
2017/04/25 13:46:02 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 229 successful
StatisticsManager 2017-04-25T13:46:02.963-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:46:02.964-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:02.975-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:46:02.986-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:46:02 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:46:02 UPR_STREAMREQ for vb 329 successful
CapiNozzle 2017-04-25T13:46:02.991-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
2017/04/25 13:46:03 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 131 successful
CapiNozzle 2017-04-25T13:46:03.066-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:46:03 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 142 successful
CapiNozzle 2017-04-25T13:46:03.085-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:46:03 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 249 successful
CapiNozzle 2017-04-25T13:46:03.115-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:46:03.128-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
2017/04/25 13:46:03 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 260 successful
CapiNozzle 2017-04-25T13:46:03.178-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:03.192-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:46:03.226-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:46:03.227-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:46:03.227-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:46:03.227-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries]
ReplicationManager 2017-04-25T13:46:03.227-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries]
PipelineManager 2017-04-25T13:46:03.244-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:46:03.244-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:46:03.246-05:00 [INFO] err_list=[{"time":"2017-04-25T13:46:03.246438425-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries"}]
PipelineManager 2017-04-25T13:46:03.246-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:46:03 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 64 successful
CapiNozzle 2017-04-25T13:46:03.272-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:03.272-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:46:03.272-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:46:03.272-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:03.272-05:00 [INFO] Send batch count=500
2017/04/25 13:46:03 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 289 successful
CheckpointManager 2017-04-25T13:46:03.335-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
CheckpointManager 2017-04-25T13:46:03.335-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:46:03.350-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:03.350-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:03.351-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:03.351-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:46:03 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 97 successful
CapiNozzle 2017-04-25T13:46:03.371-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:46:03.371-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:46:03 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:46:03 UPR_STREAMREQ for vb 294 successful
DcpNozzle 2017-04-25T13:46:03.457-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:46:03.477-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:46:03.630-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:46:03.654-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:46:03.773-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
StatisticsManager 2017-04-25T13:46:03.788-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:46:03.788-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229 {"Errors": "[{\"time\":\"2017-04-25T13:46:03.246438425-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 22, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 3612, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 22, "set_filtered": 0, "set_received_from_dcp": 3612, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:46:03.788-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:46:03.998-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:46:04.011-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:46:04.092-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:04.108-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:46:04.353-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:46:04.353-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:46:04.353-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:46:04.353-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:04.353-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:04.494-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:04.494-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:04.514-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:46:04.628-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:04.628-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:46:04.628-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:46:04.628-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:04.628-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:04.716-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:04.716-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:04.730-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:46:04.927-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:04.956-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:46:05.293-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:46:05.309-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:46:05.528-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:05.539-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:46:05.759-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:46:05.771-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:46:05.924-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:05.924-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:05.924-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:05.924-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:05.925-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:06.056-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:06.056-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:06.069-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:46:06.069-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:46:06.069-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:46:06.069-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:06.069-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:06.085-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:46:06.129-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:06.129-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:06.140-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:46:06.347-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:46:06.377-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:46:06.574-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:06.606-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:46:06.761-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:46:06.783-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:46:06.917-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:06.951-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:46:07.217-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:46:07.217-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:46:07.217-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:46:07.217-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:07.217-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:07.277-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:07.277-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:46:07.277-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:46:07.277-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:07.278-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:07.285-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:07.285-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:07.311-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:46:07.348-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:07.349-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:07.376-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:46:07.646-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:07.667-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:46:07.904-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:46:07.935-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:46:08.170-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:08.190-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:46:08.287-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:46:08.306-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:46:08.578-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:08.578-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:46:08.578-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:46:08.578-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:08.578-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:08.651-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:08.651-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:08.671-05:00 [INFO] Retrying update docs for vb 205 for the 1th time
CapiNozzle 2017-04-25T13:46:08.759-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:46:08.759-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:46:08.759-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:46:08.759-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:08.759-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:08.827-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:08.827-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:08.852-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:46:09.039-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:46:09.054-05:00 [INFO] Retrying update docs for vb 205 for the 2th time
CapiNozzle 2017-04-25T13:46:09.216-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:46:09.455-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:46:09.456-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:46:09.480-05:00 [INFO] Retrying update docs for vb 205 for the 3th time
CapiNozzle 2017-04-25T13:46:09.729-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:46:09.746-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
CapiNozzle 2017-04-25T13:46:09.878-05:00 [ERROR] batchUpdateDocs for vb 205 failed with err receieved unexpected status code, 500, from update docs request for vb 205
.
CapiNozzle 2017-04-25T13:46:09.878-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 205 after 3 retries
CapiNozzle 2017-04-25T13:46:09.878-05:00 [ERROR] Raise error condition batch update docs failed for vb 205 after 3 retries
GenericSupervisor 2017-04-25T13:46:09.878-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:09.878-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:10.001-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:10.002-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:10.015-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:46:10.108-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:46:10.108-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 4 after 3 retries
CapiNozzle 2017-04-25T13:46:10.108-05:00 [ERROR] Raise error condition batch update docs failed for vb 4 after 3 retries
GenericSupervisor 2017-04-25T13:46:10.108-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:10.108-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:10.203-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:10.204-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:10.219-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:46:10.627-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:46:10.646-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:46:10.910-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:10.931-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:46:11.111-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:46:11.129-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:46:11.424-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:11.450-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:46:11.647-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:46:11.647-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:46:11.647-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:46:11.647-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:11.647-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:11.732-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:11.732-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:11.745-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
GenericSupervisor 2017-04-25T13:46:11.785-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:46:11.785-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:46:11.785-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:46:11.855-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:11.855-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:46:11.855-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:46:11.855-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:11.855-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:11.932-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:11.933-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:11.948-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:46:12.003-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:12.022-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
ToplogyChangeDetector 2017-04-25T13:46:12.082-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:46:12.519-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:12.534-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:46:12.660-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:12.689-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
PipelineManager 2017-04-25T13:46:13.258-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries 
PipelineManager 2017-04-25T13:46:13.258-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:46:13.258-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:13.258-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
PipelineRuntimeCtx 2017-04-25T13:46:13.258-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:46:13.258-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:46:13.258-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:13.258-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:46:13.258-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:46:13.258-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=3
CapiNozzle 2017-04-25T13:46:13.258-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:13.267-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
GenericSupervisor 2017-04-25T13:46:13.268-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:46:13.268-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:13.258607233 -0500 CDT
GenericSupervisor 2017-04-25T13:46:13.275-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:13.275-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:46:13.280-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:13.280-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:46:13.280-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:13.280-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:46:13.288-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:46:13.288-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:46:13.288-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:46:13.288-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-749319229 has been stopped
GenericPipeline 2017-04-25T13:46:13.288-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-749319229 is stopped
PipelineManager 2017-04-25T13:46:13.288-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:46:13.288-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:03.246438425-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 203 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:46:13.288-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:13.289-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
GenericPipeline 2017-04-25T13:46:13.289-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
GenericPipeline 2017-04-25T13:46:13.289-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
GenericPipeline 2017-04-25T13:46:13.289-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-749319229
AsyncListener 2017-04-25T13:46:13.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:13.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:46:13.290-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:13.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:46:13.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:46:13.290-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:13.290-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:13.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:13.290543318 -0500 CDT
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:13.290741967 -0500 CDT
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:46:13.290-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.291-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:13.294-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:13.294-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
2017/04/25 13:46:13 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:46:13.309-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:46:13 Stream Closed for vb 1 StreamEnd simulated
DcpNozzle 2017-04-25T13:46:13.309-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:13.309799618 -0500 CDT
DcpNozzle 2017-04-25T13:46:13.309-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:46:13.309-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:13.309-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:46:13.310-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:13.310-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:13.31064156 -0500 CDT
DcpNozzle 2017-04-25T13:46:13.310-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:46:13.310-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:13.310-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:46:13.326-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:46:13.329-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:46:13.329-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
GenericPipeline 2017-04-25T13:46:13.351-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:13.351-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:46:13.356-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:46:13.356-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:46:13.356-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:13.356-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:46:13.440-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:13.440-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:46:13.441-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:46:13.471-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:13.472-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:46:13.472-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:13.472-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:46:13.472-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:13.472-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:46:13.472-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:46:13.473-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:46:13.473-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
GenericSupervisor 2017-04-25T13:46:13.474-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:46:13.474-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
GenericSupervisor 2017-04-25T13:46:13.474-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:13.474-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:13.474-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:13.474-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:13.474-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:46:13.475-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
CheckpointManager 2017-04-25T13:46:13.475-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
CapiUtils 2017-04-25T13:46:13.497-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:13.497-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:13.498-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:46:13.498-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
StatisticsManager 2017-04-25T13:46:13.498-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:46:13.498-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
XDCRFactory 2017-04-25T13:46:13.498-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:46:13.499-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027 is constructed, start it
GenericPipeline 2017-04-25T13:46:13.499-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[52 108 26 53 168 15 98 29 55 39 95 30 43 79 114 144 23 25 150 38 5 13 56 139 27 75 152 57 63 104 112 0 20 48 82 164 67 97 161 46 59 103 94 146 50 88 138 99 101 143 28 85 87 92 142 11 16 136 135 148 10 18 49 64 22 47 72 105 131 155 51 166 163 165 3 31 76 129 137 96 123 6 117 162 8 33 121 170 145 109 141 1 7 19 80 54 69 9 130 124 151 21 74 83 128 167 93 42 86 147 2 12 140 169 60 68 71 77 90 107 157 17 102 160 78 122 132 44 110 41 100 156 14 36 4 66 111 116 159 34 158 81 37 58 120 133 153 32 62 65 70 113 119 125 35 45 61 84 89 91 40 73 127 134 115 126 154 24 106 118 149] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[222 258 276 305 257 286 336 192 338 230 292 322 332 176 205 237 334 240 281 187 296 178 171 239 275 318 249 265 321 260 270 233 250 323 212 231 261 185 204 220 226 284 313 328 214 247 264 290 337 173 196 202 320 324 235 255 278 273 300 316 291 299 172 223 306 330 184 232 244 229 309 317 289 302 329 228 175 201 246 200 211 272 274 183 197 209 256 285 315 177 294 225 325 288 310 283 215 181 191 219 327 216 340 252 326 174 193 319 194 279 312 188 242 243 248 280 303 213 311 180 254 331 198 308 335 251 262 295 333 186 190 263 268 298 208 210 238 253 206 287 234 293 203 224 266 267 217 241 195 227 236 301 179 221 282 314 189 199 207 259 297 307 218 245 269 277 304 182 271 339]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[92 142 28 85 87 11 16 136 135 148 64 10 18 49 105 131 155 22 47 72 51 166 163 165 129 137 3 31 76 96 123 6 117 162 170 8 33 121 145 109 141 1 7 19 80 54 69 9 130 124 151 21 74 83 128 167 93 42 86 147 2 12 140 169 77 90 107 157 60 68 71 17 102 160 78 122 132 44 110 41 100 156 14 36 116 159 4 66 111 34 158 81 37 58 120 133 153 70 113 119 125 32 62 65 84 89 91 35 45 61 134 40 73 127 115 126 154 149 24 106 118 52 108 26 53 168 15 98 29 55 39 95 114 144 30 43 79 23 25 150 38 139 5 13 56 27 75 152 112 57 63 104 82 0 20 48 164 67 97 161 46 59 103 94 146 50 88 138 99 101 143] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[273 300 316 291 299 172 223 306 330 184 232 244 309 317 229 302 329 289 228 201 246 175 200 211 272 274 183 197 209 256 285 315 177 294 225 325 310 288 283 215 181 191 219 327 340 216 252 326 319 174 193 194 279 312 188 242 243 248 280 303 213 311 180 254 331 198 308 335 262 295 333 251 186 190 263 268 298 210 238 253 208 206 287 234 293 203 224 266 267 241 217 195 227 236 301 179 221 282 314 189 199 207 259 297 307 218 245 269 277 304 182 271 339 258 276 305 222 257 286 336 338 192 292 322 230 332 176 205 237 334 240 281 187 296 178 171 239 275 318 249 265 321 260 270 323 233 250 212 231 261 185 204 220 226 284 313 328 264 290 337 214 247 202 320 324 173 196 235 255 278]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: active:true worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi failure_restart_interval:10 optimistic_replication_threshold:256 log_level:Info checkpoint_interval:1800]
CheckpointManager 2017-04-25T13:46:13.499-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027...
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:13.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:46:13.510-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:13.510-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:46:13.510-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:46:13.510-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:46:13.510-05:00 [INFO] Checkpointing starts in 1800.045 sec
PipelineRuntimeCtx 2017-04-25T13:46:13.510-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:46:13.510-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:46:13.511-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:46:13.511-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:46:13.514-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:46:13.514-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:46:13.514-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:46:13.514-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:46:13.517-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:46:13.557-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:13.557-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:13.557-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:46:13.586-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:46:13.586-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
ToplogyChangeDetector 2017-04-25T13:46:13.588-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:46:13.588-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:46:13.607-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:46:13.607-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:46:13.607-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:46:13.607-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:46:13.657-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:46:13.657-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:13.657-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:46:13.657-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:46:13.657-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:46:13.657-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:46:13.686-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:46:13.686-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:46:13.686-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:46:13.703-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:46:13.703-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:46:13.706-05:00 [INFO] Got the bucket metadata for ckmgr
AsyncListener 2017-04-25T13:46:13.792-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:46:13.792-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:46:13.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:46:13.792-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:46:13.792-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:46:13.792-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:46:13.871-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:46:13.871-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:13.871-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:46:13.890-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:46:13.890-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:46:13.891-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:46:13.891-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027 is started----------
PipelineManager 2017-04-25T13:46:13.891-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:46:13.891-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:46:13.891-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:46:14.620-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:46:15.144-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:46:15.144-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:46:15.144-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:46:15.144-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:46:15 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:46:15.212-05:00 [INFO] Send batch count=102
2017/04/25 13:46:15 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:46:15.228-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:15.228-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:15.243-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:46:15.252-05:00 [INFO] Send batch count=72
CapiNozzle 2017-04-25T13:46:15.252-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:46:15 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 312 successful
CapiNozzle 2017-04-25T13:46:15.418-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:15.431-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:46:15.446-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:15.467-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:46:15 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 227 successful
StatisticsManager 2017-04-25T13:46:15.564-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=5, changes_left=315503
2017/04/25 13:46:15 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 137 successful
CapiNozzle 2017-04-25T13:46:15.590-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:15.610-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:46:15 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 231 successful
CapiNozzle 2017-04-25T13:46:15.622-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:15.650-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:46:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:46:15.770-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:15.770-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:46:15.770-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:46:15.770-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:46:15.770-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:46:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 162 successful
PipelineManager 2017-04-25T13:46:15.782-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:46:15.783-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:46:15.785-05:00 [INFO] err_list=[{"time":"2017-04-25T13:46:15.785783134-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:46:15.785-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:46:15 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 269 successful
CapiNozzle 2017-04-25T13:46:15.814-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:15.836-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:46:15.842-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:15.842-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:15.871-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:46:15 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 174 successful
CapiNozzle 2017-04-25T13:46:15.935-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:15.935-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:15.935-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:15.935-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:15.935-05:00 [INFO] Send batch count=500
2017/04/25 13:46:15 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:46:15 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:46:15.989-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:15.989-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:16.013-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:46:16 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 192 successful
CheckpointManager 2017-04-25T13:46:16.044-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
CheckpointManager 2017-04-25T13:46:16.044-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:46:16.061-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:46:16 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:46:16 UPR_STREAMREQ for vb 197 successful
CapiNozzle 2017-04-25T13:46:16.201-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
DcpNozzle 2017-04-25T13:46:16.204-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:46:16.218-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:46:16.463-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:16.486-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:46:16.516-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:46:16.516-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027 {"Errors": "[{\"time\":\"2017-04-25T13:46:15.785783134-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315503, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 23, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 73, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 5, "docs_received_from_dcp": 12291, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 73, "set_filtered": 0, "set_received_from_dcp": 12291, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:46:16.695-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
PipelineManager 2017-04-25T13:46:16.700-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:15.785783134-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:46:16.715-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:46:16.981-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:16.994-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:46:17.166-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:17.166-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:46:17.166-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:46:17.166-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:17.166-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:17.262-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:17.262-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:17.307-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:46:17.421-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:17.421-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:46:17.421-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:46:17.421-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:17.421-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:17.506-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:17.506-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:17.527-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:46:17.591-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:17.609-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:46:17.978-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:18.006-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:46:18.089-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:18.106-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:46:18.736-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:18.760-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:18.760-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:46:18.760-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:46:18.760-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:18.760-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:18.788-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:46:18.850-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:18.851-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:18.879-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:46:19.226-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:19.226-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:46:19.226-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:46:19.226-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:19.226-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:19.292-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:19.292-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:19.307-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:46:19.369-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:19.403-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:46:19.559-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:19.580-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:46:19.754-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:19.777-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:46:19.934-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:19.973-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:46:20.163-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:20.163-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:46:20.163-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:46:20.163-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:20.163-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:20.226-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:20.226-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:20.245-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:46:20.285-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:20.285-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:20.285-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:20.285-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:20.285-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:20.357-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:20.357-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:20.374-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
CapiNozzle 2017-04-25T13:46:20.499-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:20.524-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:46:20.714-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:46:20.726-05:00 [INFO] Retrying update docs for vb 208 for the 2th time
CapiNozzle 2017-04-25T13:46:20.879-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:20.899-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:46:21.100-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:46:21.120-05:00 [INFO] Retrying update docs for vb 208 for the 3th time
CapiNozzle 2017-04-25T13:46:21.237-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:21.237-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:46:21.237-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:46:21.237-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:21.237-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:21.387-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:21.387-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:21.412-05:00 [INFO] Retrying update docs for vb 6 for the 1th time
CapiNozzle 2017-04-25T13:46:21.473-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:46:21.473-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 208 after 3 retries
CapiNozzle 2017-04-25T13:46:21.473-05:00 [ERROR] Raise error condition batch update docs failed for vb 208 after 3 retries
GenericSupervisor 2017-04-25T13:46:21.473-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:21.473-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:21.571-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:21.571-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:21.590-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:46:21.909-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:46:21.936-05:00 [INFO] Retrying update docs for vb 6 for the 2th time
CapiNozzle 2017-04-25T13:46:22.151-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:46:22.172-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:46:22.443-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:46:22.460-05:00 [INFO] Retrying update docs for vb 6 for the 3th time
CapiNozzle 2017-04-25T13:46:22.589-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:46:22.606-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:46:22.954-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:46:22.954-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 6 after 3 retries
CapiNozzle 2017-04-25T13:46:22.954-05:00 [ERROR] Raise error condition batch update docs failed for vb 6 after 3 retries
GenericSupervisor 2017-04-25T13:46:22.954-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:22.954-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:23.035-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:23.035-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:23.052-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:46:23.151-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:46:23.151-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:46:23.151-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:46:23.151-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:23.151-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:23.248-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:23.248-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:23.265-05:00 [INFO] Retrying update docs for vb 308 for the 1th time
GenericSupervisor 2017-04-25T13:46:23.511-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:46:23.511-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:46:23.511-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:46:23.519-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:46:23.536-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
ToplogyChangeDetector 2017-04-25T13:46:23.789-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:46:23.879-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:46:23.890-05:00 [INFO] Retrying update docs for vb 308 for the 2th time
CapiNozzle 2017-04-25T13:46:24.222-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:46:24.232-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:46:24.481-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:46:24.492-05:00 [INFO] Retrying update docs for vb 308 for the 3th time
CapiNozzle 2017-04-25T13:46:24.693-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:46:24.693-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:46:24.693-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:46:24.693-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:24.693-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:24.775-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:24.775-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:24.786-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:46:24.984-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:46:24.984-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 308 after 3 retries
CapiNozzle 2017-04-25T13:46:24.984-05:00 [ERROR] Raise error condition batch update docs failed for vb 308 after 3 retries
GenericSupervisor 2017-04-25T13:46:24.984-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:24.984-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:25.071-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:25.071-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:25.082-05:00 [INFO] Retrying update docs for vb 309 for the 1th time
CapiNozzle 2017-04-25T13:46:25.314-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:25.324-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:46:25.425-05:00 [ERROR] batchUpdateDocs for vb 309 failed with err receieved unexpected status code, 500, from update docs request for vb 309
.
CapiNozzle 2017-04-25T13:46:25.435-05:00 [INFO] Retrying update docs for vb 309 for the 2th time
CapiNozzle 2017-04-25T13:46:25.656-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:25.667-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
PipelineManager 2017-04-25T13:46:25.786-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:46:25.786-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:46:25.786-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:25.786-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
PipelineRuntimeCtx 2017-04-25T13:46:25.786-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:46:25.786-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:25.786-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:46:25.786-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:25.786352847 -0500 CDT
GenericSupervisor 2017-04-25T13:46:25.786-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:25.786-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:46:25.786-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:25.786-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:46:25.786-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:25.786-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:46:25.786-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:46:25.786-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:46:25.786-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:46:25.786-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-472441027 has been stopped
GenericPipeline 2017-04-25T13:46:25.786-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-472441027 is stopped
PipelineManager 2017-04-25T13:46:25.786-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:46:25.787-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:15.785783134-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:46:25.787-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:46:25.787-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:46:25.787-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:46:25.787-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
GenericPipeline 2017-04-25T13:46:25.787-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-472441027
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.787-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:46:25.788-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:46:25.788-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:25.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
CapiNozzle 2017-04-25T13:46:25.788-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:46:25.788-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:25.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:46:25.788-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:46:25.788-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:25.788-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:25.788-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:25.788397223 -0500 CDT
DcpNozzle 2017-04-25T13:46:25.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:46:25.788-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:25.78819672 -0500 CDT
DcpNozzle 2017-04-25T13:46:25.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:46:25.789-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:46:25.789-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:46:25.789-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.789-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:46:25.789-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:46:25.789-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.789-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:46:25.789-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:25.789-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:46:25.789-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:46:25.789-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:46:25.789-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.789-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:46:25.789-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:46:25.790-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:25.790-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:46:25.790-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:25.790-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:46:25.790-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:25.790-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:25.790826023 -0500 CDT
DcpNozzle 2017-04-25T13:46:25.790-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:46:25.790-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:25.790-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:46:25.793-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:25.793-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:25.793214761 -0500 CDT
DcpNozzle 2017-04-25T13:46:25.793-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:46:25.793-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:25.793-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:46:25 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:46:25 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:46:25 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:46:25 Stream Closed for vb 172 StreamEnd simulated
XDCRFactory 2017-04-25T13:46:25.908-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:25.909-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:46:25.909-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:46:26.061-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:26.061-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:46:26.062-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:26.062-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:46:26.062-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:26.062-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:46:26.063-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:46:26.063-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:46:26.064-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
GenericSupervisor 2017-04-25T13:46:26.066-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:46:26.066-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
GenericSupervisor 2017-04-25T13:46:26.066-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:26.066-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:26.066-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:26.066-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:26.066-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:46:26.066-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
CheckpointManager 2017-04-25T13:46:26.066-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
CapiUtils 2017-04-25T13:46:26.181-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:26.181-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:26.181-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:46:26.182-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
StatisticsManager 2017-04-25T13:46:26.183-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:46:26.183-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
XDCRFactory 2017-04-25T13:46:26.183-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:46:26.183-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210 is constructed, start it
GenericPipeline 2017-04-25T13:46:26.184-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[320 223 183 202 204 216 219 226 271 294 184 194 244 314 283 235 253 195 254 256 340 187 190 213 290 326 209 291 312 197 263 222 306 311 262 292 309 199 255 258 336 211 243 273 232 272 289 298 180 221 229 238 269 316 268 179 322 324 334 296 335 318 215 239 330 196 182 241 261 171 205 265 279 325 185 192 327 304 317 224 299 248 278 333 242 173 198 323 331 277 328 176 240 313 339 175 233 234 300 189 236 260 303 177 250 266 329 310 214 275 220 267 210 276 305 193 230 293 200 191 227 231 270 338 284 319 174 281 297 308 321 301 172 178 237 302 228 264 280 217 225 286 315 332 337 181 186 188 307 247 207 246 295 252 203 249 257 288 282 287 245 208 206 218 201 212 251 259 285 274] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[7 69 16 55 162 153 83 129 151 142 106 5 84 93 139 158 24 36 110 169 140 6 26 97 95 146 112 125 128 8 47 104 38 105 31 39 143 11 51 88 35 32 152 74 156 28 65 96 121 72 87 108 14 25 58 2 148 159 23 42 57 109 118 166 170 50 70 85 145 15 123 103 137 147 41 78 90 3 27 18 64 89 141 60 98 111 132 149 48 91 157 77 43 161 99 116 29 155 160 138 165 67 130 134 9 150 63 13 22 56 45 82 126 46 81 101 33 163 168 79 119 34 53 62 167 76 86 131 19 154 68 100 133 59 114 164 113 120 4 30 61 1 21 135 122 40 71 92 124 127 144 0 94 107 20 73 115 37 52 66 54 10 17 44 75 80 102 136 12 49 117]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[163 168 33 119 79 53 62 34 86 131 167 76 154 19 100 133 68 114 164 59 30 61 113 120 4 21 135 1 71 92 122 40 127 144 124 94 107 0 20 52 66 73 115 37 17 44 54 10 80 102 75 49 117 136 12 69 7 55 162 16 129 151 153 83 142 84 93 106 5 36 110 139 158 24 169 26 97 140 6 146 95 47 104 112 125 128 8 105 38 39 31 51 88 143 11 35 152 32 156 74 65 28 121 96 25 58 72 87 108 14 148 2 42 57 159 23 70 85 109 118 166 170 50 145 123 15 78 90 103 137 147 41 3 27 64 89 18 141 60 98 132 149 111 91 157 48 77 161 43 116 99 155 160 29 130 134 138 165 67 150 9 22 56 63 13 82 45 81 101 126 46] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[174 281 297 308 301 321 172 178 237 302 217 228 264 280 225 286 315 332 337 181 186 188 307 247 207 246 295 252 203 249 257 288 282 287 245 208 206 218 201 212 251 259 285 274 223 320 183 202 204 216 219 271 294 226 184 194 244 314 235 253 283 195 254 256 340 187 190 213 326 290 291 312 209 197 263 222 306 311 255 258 262 292 309 199 243 273 336 211 180 232 272 289 298 221 229 238 269 316 179 268 296 322 324 334 318 335 196 215 239 330 182 241 171 261 205 265 279 325 185 327 192 224 299 304 317 242 248 278 333 173 198 323 331 277 328 176 175 233 240 313 339 234 300 189 236 260 303 177 250 266 329 214 275 310 220 210 267 276 305 193 230 293 200 191 227 231 270 284 338 319]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 replication_type:capi filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 log_level:Info doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:46:26.184-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210...
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:26.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:46:26.192-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:26.193-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:46:26.193-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:46:26.193-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:46:26.193-05:00 [INFO] Checkpointing starts in 1802.165 sec
PipelineRuntimeCtx 2017-04-25T13:46:26.193-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:46:26.193-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:46:26.193-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:46:26.193-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:46:26.195-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:46:26.195-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:46:26.195-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:46:26.195-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:46:26.196-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:46:26.221-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:46:26.221-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:46:26.221-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:46:26.241-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:46:26.246-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:46:26.246-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:46:26.277-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:46:26.283-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:46:26.283-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:26.283-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:46:26.283-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:46:26.283-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:46:26.283-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:46:26.306-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:26.306-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:26.306-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:46:26.321-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:46:26.321-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:46:26.321-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:46:26.337-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:46:26.337-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:46:26.362-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:46:26.470-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:46:26.471-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:26.471-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:46:26.499-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:46:26.499-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:46:26.499-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:46:26.499-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210 is started----------
PipelineManager 2017-04-25T13:46:26.499-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:46:26.499-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:46:26.499-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:46:26.528-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:46:26.754-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:46:26.755-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:46:26.755-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:46:26.755-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:46:26 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:46:26.867-05:00 [INFO] Send batch count=122
CapiNozzle 2017-04-25T13:46:26.869-05:00 [INFO] Send batch count=157
CapiNozzle 2017-04-25T13:46:26.903-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:26.903-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:46:26 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:46:26 UPR_STREAMREQ for vb 106 successful
CapiNozzle 2017-04-25T13:46:26.945-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:46:26.962-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:26.964-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:26.979-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:46:27 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 27 successful
CapiNozzle 2017-04-25T13:46:27.243-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:46:27 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 327 successful
CapiNozzle 2017-04-25T13:46:27.269-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:46:27 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 334 successful
StatisticsManager 2017-04-25T13:46:27.331-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:46:27.334-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:27.418-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:46:27 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 150 successful
CapiNozzle 2017-04-25T13:46:27.473-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:27.488-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
2017/04/25 13:46:27 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:46:27 UPR_STREAMREQ for vb 175 successful
CapiNozzle 2017-04-25T13:46:28.306-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:28.306-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:46:28.306-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:46:28.306-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
ReplicationManager 2017-04-25T13:46:28.307-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
2017/04/25 13:46:28 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 271 successful
CapiNozzle 2017-04-25T13:46:28.320-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:46:28 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 281 successful
PipelineManager 2017-04-25T13:46:28.373-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:46:28.373-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:46:28.376-05:00 [INFO] err_list=[{"time":"2017-04-25T13:46:28.376525515-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]
PipelineManager 2017-04-25T13:46:28.376-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:46:28.409-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:46:28.412-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:46:28 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 86 successful
CapiNozzle 2017-04-25T13:46:28.454-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:28.454-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:28.470-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:46:28 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 96 successful
CheckpointManager 2017-04-25T13:46:28.632-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
CheckpointManager 2017-04-25T13:46:28.632-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:46:28 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 290 successful
CapiNozzle 2017-04-25T13:46:28.693-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
2017/04/25 13:46:28 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:46:28 UPR_STREAMREQ for vb 299 successful
CapiNozzle 2017-04-25T13:46:28.709-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:46:28.779-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:28.779-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:28.779-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:28.779-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:28.779-05:00 [INFO] Send batch count=500
DcpNozzle 2017-04-25T13:46:28.788-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:46:28.827-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:46:28.864-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:28.866-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:28.896-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:46:28.914-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:28.962-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
StatisticsManager 2017-04-25T13:46:29.218-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:46:29.218-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210 {"Errors": "[{\"time\":\"2017-04-25T13:46:28.376525515-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 105, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 28025, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 105, "set_filtered": 0, "set_received_from_dcp": 28030, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:46:29.373-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:29.389-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:46:29.445-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:29.445-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:46:29.445-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:46:29.445-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:29.445-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:29.544-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:29.544-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:29.565-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:46:29.837-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:29.848-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:46:30.072-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:30.087-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:46:30.326-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:46:30.327-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:46:30.327-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:46:30.327-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:30.327-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:30.412-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:30.412-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:30.432-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:46:30.515-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:30.535-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:46:30.718-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:30.765-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:46:30.925-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:30.925-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:46:30.925-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:46:30.925-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:30.925-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:30.997-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:31.029-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:46:31.031-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:31.031-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:31.056-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:46:31.235-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:31.235-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:31.236-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:31.236-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:31.236-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:31.305-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:31.306-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:31.327-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:46:31.506-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:31.554-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:46:31.671-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
PipelineManager 2017-04-25T13:46:31.680-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:28.376525515-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]}
]
CapiNozzle 2017-04-25T13:46:31.696-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:46:31.927-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:31.944-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:46:32.058-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:32.081-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:46:32.349-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:32.349-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:46:32.349-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:46:32.349-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:32.349-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:32.426-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:32.426-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:32.443-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:46:32.469-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:32.469-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:46:32.469-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:46:32.469-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:32.469-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:32.527-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:32.527-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:32.540-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:46:32.741-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:32.760-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:46:32.877-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:46:32.892-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:46:33.125-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:33.145-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:46:33.253-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:46:33.269-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:46:33.545-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:33.545-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:46:33.545-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:46:33.545-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:33.545-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:33.702-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:33.702-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:33.719-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:46:33.831-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:46:33.831-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:46:33.831-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:46:33.831-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:33.831-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:33.897-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:33.897-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:33.909-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:46:34.086-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:46:34.111-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:46:34.346-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:34.360-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:46:34.624-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:46:34.649-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:46:34.857-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:34.868-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:46:35.031-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:46:35.031-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:46:35.031-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:46:35.031-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:35.031-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:35.115-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:35.115-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:35.126-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:46:35.242-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:35.242-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:46:35.242-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:46:35.242-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:35.242-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:35.350-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:35.350-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:35.361-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:46:35.437-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:46:35.448-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:46:36.028-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:46:36.039-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
GenericSupervisor 2017-04-25T13:46:36.193-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:46:36.193-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:46:36.193-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:46:36.194-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:46:36.205-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
ToplogyChangeDetector 2017-04-25T13:46:36.451-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:46:36.505-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:46:36.516-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:46:36.708-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:46:36.708-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:46:36.708-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:46:36.708-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:36.708-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:36.803-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:36.803-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:36.814-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:46:37.299-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:46:37.299-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:46:37.300-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:46:37.300-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:37.300-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:37.379-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:37.379-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:37.391-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:46:37.456-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:37.467-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:46:37.723-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:46:37.734-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:46:37.977-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:46:37.987-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:46:38.271-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:46:38.282-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
PipelineManager 2017-04-25T13:46:38.376-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries 
PipelineManager 2017-04-25T13:46:38.376-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:46:38.376-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:38.376-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
PipelineRuntimeCtx 2017-04-25T13:46:38.376-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:46:38.376-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:38.377-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:46:38.377-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:38.377004946 -0500 CDT
GenericSupervisor 2017-04-25T13:46:38.377-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:38.377-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:46:38.377-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:38.377-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:46:38.377-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:38.377-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:46:38.377-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:46:38.377-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:46:38.377-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:46:38.377-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-63067210 has been stopped
GenericPipeline 2017-04-25T13:46:38.377-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-63067210 is stopped
PipelineManager 2017-04-25T13:46:38.377-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:46:38.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
PipelineManager 2017-04-25T13:46:38.377-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:28.376525515-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:46:38.377-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:38.377-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
GenericPipeline 2017-04-25T13:46:38.377-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
GenericPipeline 2017-04-25T13:46:38.377-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
GenericPipeline 2017-04-25T13:46:38.377-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-63067210
AsyncListener 2017-04-25T13:46:38.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:38.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:46:38.378-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:46:38.378-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:38.378-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:38.378-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:38.378-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:46:38.378-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:46:38.378-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:46:38.378-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:38.378-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:46:38.378-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:38.378734327 -0500 CDT
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:46:38.378-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:38.378466819 -0500 CDT
DcpNozzle 2017-04-25T13:46:38.378-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:46:38.378-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.379-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:38.379-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.379-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:38.379-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:38.379-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:46:38.379-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:46:38.379-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:46:38.380-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:46:38.380-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:46:38.380-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:46:38.380-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:46:38.380-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:38.380-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:46:38.380-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:38.380-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:46:38.380-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:38.380-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:38.380542998 -0500 CDT
DcpNozzle 2017-04-25T13:46:38.380-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:46:38.380-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:38.380-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:46:38.381-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:38.381-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:38.38186659 -0500 CDT
DcpNozzle 2017-04-25T13:46:38.381-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:46:38.381-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:38.381-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:46:38 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:46:38 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:46:38 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:46:38.507-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:38.507-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:46:38.508-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:46:38.618-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:38.618-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:46:38.619-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:38.619-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:46:38.619-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:38.619-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:46:38.619-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:46:38.619-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:46:38.620-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
GenericSupervisor 2017-04-25T13:46:38.622-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:46:38.622-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
GenericSupervisor 2017-04-25T13:46:38.622-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:38.622-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:38.622-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:38.622-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:38.622-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:46:38.622-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
CheckpointManager 2017-04-25T13:46:38.622-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
CapiUtils 2017-04-25T13:46:38.649-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:38.649-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:38.649-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:46:38.650-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
StatisticsManager 2017-04-25T13:46:38.651-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:46:38.651-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
XDCRFactory 2017-04-25T13:46:38.651-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:46:38.651-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872 is constructed, start it
GenericPipeline 2017-04-25T13:46:38.652-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[7 11 17 53 85 61 108 128 126 138 139 54 153 59 29 74 25 41 55 51 78 37 72 96 103 162 45 62 20 47 120 36 92 98 156 127 66 160 58 140 81 164 57 60 122 90 116 161 166 2 100 152 170 21 71 117 150 12 79 40 86 131 83 102 124 10 39 101 112 129 163 5 43 149 99 23 111 31 38 68 158 28 121 133 14 104 155 13 26 89 168 169 34 115 132 67 105 73 91 137 6 9 18 49 77 63 159 167 88 0 27 30 80 165 70 87 136 35 42 107 82 147 145 1 56 106 144 3 8 15 46 110 157 22 32 75 134 69 84 148 19 52 114 65 95 109 118 119 154 48 113 142 4 33 151 97 143 24 50 93 94 123 44 141 146 64 76 125 16 130 135] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[235 325 177 301 284 297 324 191 209 221 293 188 216 204 259 178 186 199 210 211 237 242 250 248 334 224 227 241 256 273 287 326 207 309 294 213 258 270 331 247 201 202 263 312 180 260 183 185 274 283 245 282 320 236 280 339 187 198 307 338 223 231 233 257 313 214 292 190 264 240 310 317 172 304 337 181 219 239 179 225 215 321 192 268 272 261 299 305 176 230 203 205 228 252 300 222 238 254 220 243 285 291 173 182 189 194 333 206 249 318 295 323 184 197 303 234 290 311 271 281 171 267 232 329 218 244 315 200 262 266 269 319 255 276 174 314 193 251 289 330 327 332 278 288 196 279 286 302 253 265 275 298 195 296 306 335 340 229 316 328 217 322 226 336 246 308 175 208 212 277]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[70 87 136 35 42 107 82 147 145 1 56 106 144 3 8 15 46 110 157 22 32 75 134 69 84 148 19 52 114 65 95 109 118 119 154 48 113 142 4 33 151 97 143 24 50 93 94 123 44 141 146 64 76 125 16 130 135 7 11 17 53 85 61 108 128 126 138 139 54 153 59 29 74 25 41 55 51 78 37 72 96 103 162 45 62 20 47 120 36 92 98 156 127 66 160 58 140 81 164 57 60 122 90 116 161 166 2 100 152 170 21 71 117 150 12 79 40 86 131 83 102 124 10 39 101 112 129 163 5 43 149 99 23 111 31 38 68 158 28 121 133 14 104 155 13 26 89 168 169 34 115 132 67 105 73 91 137 6 9 18 49 77 63 159 167 88 0 27 30 80 165] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[234 290 311 271 281 171 267 329 232 218 244 315 319 200 262 266 269 255 276 174 314 193 251 289 330 327 332 278 288 196 279 286 302 253 265 275 298 306 195 296 335 340 229 316 328 217 322 226 336 246 308 175 208 212 277 235 325 177 301 284 297 324 191 209 221 293 188 216 204 259 178 186 199 210 211 237 242 250 248 334 287 224 227 241 256 273 326 207 309 294 213 258 270 331 247 201 202 263 312 180 260 283 183 185 274 245 282 320 236 280 339 187 198 307 338 223 231 233 257 313 214 292 190 264 310 240 317 172 304 337 181 219 239 179 225 215 321 192 268 272 261 299 305 176 230 203 205 228 252 300 222 238 254 220 243 285 291 333 173 182 189 194 206 249 318 295 323 303 184 197]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 active:true optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:46:38.652-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872...
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:38.659-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:46:38.659-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:46:38.659-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:46:38.659-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:38.659-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:46:38.659-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:46:38.659-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:46:38.659-05:00 [INFO] Checkpointing starts in 1801.537 sec
PipelineRuntimeCtx 2017-04-25T13:46:38.659-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:46:38.659-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:46:38.659-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:46:38.660-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:46:38.661-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:46:38.661-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:46:38.665-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:46:38.688-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:46:38.698-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:46:38.698-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:46:38.698-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:46:38.706-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:38.706-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:38.706-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:46:38.713-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:46:38.713-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:46:38.720-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:46:38.722-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:46:38.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:46:38.729-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:38.729-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:46:38.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:46:38.729-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:46:38.729-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:46:38.758-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:46:38.758-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:46:38.758-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:46:38.783-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:46:38.783-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:46:38.855-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:46:38.908-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:46:38.908-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:38.908-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:46:38.908-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:46:38.908-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:46:38.908-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:46:38.909-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872 is started----------
PipelineManager 2017-04-25T13:46:38.909-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:46:38.909-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:46:38.909-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:46:39.629-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:46:39.630-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:46:39.630-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:46:39.630-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:46:39 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 100 successful
StatisticsManager 2017-04-25T13:46:39.732-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:46:39.733-05:00 [INFO] Send batch count=71
2017/04/25 13:46:39 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:46:39.745-05:00 [INFO] Send batch count=374
CapiNozzle 2017-04-25T13:46:39.778-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:39.778-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:39.791-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:46:39 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:46:39.831-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:39.831-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:46:39 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 5 successful
CapiNozzle 2017-04-25T13:46:39.856-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:46:39 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:46:39 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 119 successful
CapiNozzle 2017-04-25T13:46:40.111-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:40.149-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
2017/04/25 13:46:40 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 146 successful
CapiNozzle 2017-04-25T13:46:40.376-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:40.395-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:46:40 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 83 successful
CapiNozzle 2017-04-25T13:46:40.616-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
2017/04/25 13:46:40 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 193 successful
CapiNozzle 2017-04-25T13:46:40.629-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
2017/04/25 13:46:40 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 89 successful
CheckpointManager 2017-04-25T13:46:40.670-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
CheckpointManager 2017-04-25T13:46:40.670-05:00 [INFO] Done with SetVBTimestamps
StatisticsManager 2017-04-25T13:46:40.713-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:46:40 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:46:40 UPR_STREAMREQ for vb 97 successful
DcpNozzle 2017-04-25T13:46:40.809-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:46:40.831-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:46:40.847-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:40.865-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:46:40.936-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:40.936-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:46:40.936-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:46:40.936-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:46:40.936-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
PipelineManager 2017-04-25T13:46:40.941-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:46:40.941-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:46:40.944-05:00 [INFO] err_list=[{"time":"2017-04-25T13:46:40.944211359-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:46:40.947-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:46:41.055-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:41.056-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:41.074-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:46:41.426-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:41.426-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:41.426-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:41.426-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:41.426-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:41.484-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:41.484-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:42.505-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:46:42.506-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:46:42.525-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:46:42.525-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872 {"Errors": "[{\"time\":\"2017-04-25T13:46:40.944211359-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 217, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 39912, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 217, "set_filtered": 0, "set_received_from_dcp": 39912, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:46:42.557-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:46:43.222-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:43.239-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:46:43.516-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:43.528-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:46:44.084-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:46:44.084-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:46:44.084-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:46:44.084-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:44.084-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:44.159-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:44.159-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:44.178-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:46:44.500-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:44.518-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:46:45.011-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:45.023-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:46:45.288-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:45.288-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:45.288-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:45.288-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:45.288-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:45.357-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:45.357-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:45.379-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:46:45.771-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:45.783-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:46:45.977-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:45.998-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:46:46.702-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:46.702-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:46:46.702-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:46:46.702-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:46.702-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:46.790-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:46.790-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:46:46.798-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:40.944211359-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]}
]
CapiNozzle 2017-04-25T13:46:46.805-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
ReplicationManager 2017-04-25T13:46:46.974-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:46:47.013-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:47.027-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:46:47.713-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:47.726-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:46:48.089-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:48.089-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:46:48.089-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:46:48.089-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:48.089-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:48.223-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:48.223-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:48.234-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
GenericSupervisor 2017-04-25T13:46:48.660-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:46:48.660-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:46:48.660-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:46:48.891-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:46:49.430-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:49.446-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:46:49.648-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:49.659-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:46:50.185-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:46:50.185-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:46:50.185-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:46:50.185-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:50.186-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:50.287-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:50.287-05:00 [ERROR] batchUpdateDocs for vb 7 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:50.298-05:00 [INFO] Retrying update docs for vb 7 for the 1th time
CapiNozzle 2017-04-25T13:46:50.537-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:46:50.547-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
PipelineManager 2017-04-25T13:46:50.947-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:46:50.948-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:46:50.948-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:50.948-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
PipelineRuntimeCtx 2017-04-25T13:46:50.948-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:46:50.948-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:50.948-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:46:50.948-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:50.948276817 -0500 CDT
GenericSupervisor 2017-04-25T13:46:50.948-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:50.948-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:46:50.948-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:50.948-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:46:50.948-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:46:50.948-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:46:50.948-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:46:50.948-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:46:50.948-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:46:50.948-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-619334872 has been stopped
GenericPipeline 2017-04-25T13:46:50.948-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-619334872 is stopped
PipelineManager 2017-04-25T13:46:50.948-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:46:50.948-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:40.944211359-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:46:50.949-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:46:50.949-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
GenericPipeline 2017-04-25T13:46:50.949-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
GenericPipeline 2017-04-25T13:46:50.949-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
GenericPipeline 2017-04-25T13:46:50.949-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-619334872
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:46:50.949-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:50.950-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:46:50.950-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:50.950-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:46:50.950-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:50.950-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:46:50.950-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:46:50.950-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:46:50.950-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:46:50.950-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:46:50.950-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:46:50.950-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:50.950435113 -0500 CDT
DcpNozzle 2017-04-25T13:46:50.951-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:46:50.951-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:50.950740961 -0500 CDT
CapiNozzle 2017-04-25T13:46:50.951-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:46:50.951-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:46:50.952-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:46:50.952-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:46:50.952-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:46:50.952-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:50.952-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:46:50.953-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:46:50.953-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:50.953-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:46:50.953-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:50.953-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:46:50.957-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:50.957-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:46:50.957-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:50.957603838 -0500 CDT
DcpNozzle 2017-04-25T13:46:50.957-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:46:50.957622685 -0500 CDT
DcpNozzle 2017-04-25T13:46:50.957-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:46:50.957-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:46:50.957-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:50.957-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:46:50.957-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:46:50.957-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:46:50 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:46:50 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:46:51.104-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:51.105-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:46:51.105-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:46:51.188-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:46:51.189-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:46:51.189-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:51.189-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:46:51.189-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:46:51.189-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:46:51.189-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:46:51.190-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:46:51.190-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
GenericSupervisor 2017-04-25T13:46:51.192-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:46:51.192-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
GenericSupervisor 2017-04-25T13:46:51.192-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:51.192-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:51.192-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:51.192-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:46:51.192-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:46:51.192-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
CheckpointManager 2017-04-25T13:46:51.192-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
CapiUtils 2017-04-25T13:46:51.277-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:51.277-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:51.277-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:46:51.278-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
StatisticsManager 2017-04-25T13:46:51.278-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:46:51.278-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
XDCRFactory 2017-04-25T13:46:51.278-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:46:51.279-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962 is constructed, start it
GenericPipeline 2017-04-25T13:46:51.280-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[32 35 7 110 85 46 94 165 103 150 11 22 44 49 123 24 30 84 89 117 68 77 82 151 39 54 153 3 122 41 100 128 146 36 113 154 163 51 157 168 12 42 88 125 130 14 140 161 17 65 166 149 116 131 26 86 126 148 25 31 67 96 106 55 141 164 118 20 121 95 4 99 124 147 109 23 59 136 47 58 75 87 93 108 8 162 57 129 145 15 80 102 107 139 0 56 1 40 37 92 144 160 5 27 71 10 50 63 64 105 18 91 111 38 60 156 6 21 13 61 83 114 127 134 155 112 133 28 34 53 78 120 45 73 98 119 29 66 69 72 137 97 135 158 159 167 2 33 104 81 152 169 79 90 170 52 101 70 19 62 76 115 9 132 43 48 74 138 142 16 143] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[253 268 327 224 316 322 326 221 222 259 171 292 180 194 304 274 340 235 332 257 282 174 227 200 333 190 233 250 330 213 280 231 247 278 294 187 243 276 216 269 176 228 285 204 287 293 303 320 197 205 215 251 238 172 318 179 196 271 201 211 252 286 306 188 297 308 338 186 223 290 310 319 195 265 184 237 177 298 325 208 229 300 311 181 264 295 321 241 261 273 296 182 256 331 199 209 240 242 245 275 214 263 193 202 317 210 212 277 218 302 314 203 262 185 307 337 334 335 189 219 267 336 312 183 315 324 288 339 173 239 272 220 255 207 289 244 305 254 284 206 226 230 328 270 291 329 178 175 225 258 283 191 246 279 192 266 309 260 217 313 323 198 236 232 234 299 301 248 249 281]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[57 129 145 139 15 80 102 107 0 56 1 40 37 92 144 160 5 27 71 105 10 50 63 64 18 91 111 38 60 156 6 21 13 61 83 114 127 134 155 112 133 120 28 34 53 78 45 73 98 119 137 29 66 69 72 97 135 158 159 167 2 33 104 81 152 169 79 90 170 52 101 70 19 62 76 115 9 132 43 48 74 138 142 16 143 32 35 7 110 85 46 94 165 103 150 11 22 44 49 123 117 24 30 84 89 68 77 82 151 39 54 153 3 122 41 100 128 146 36 113 154 163 51 157 168 12 42 88 125 130 14 140 161 17 65 166 149 116 131 26 86 126 148 106 25 31 67 96 55 141 164 118 20 121 95 4 99 124 147 109 23 59 93 108 136 47 58 75 87 8 162] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[182 199 256 331 209 240 275 242 245 193 202 214 263 212 277 317 210 218 302 314 185 203 262 307 337 219 267 334 335 189 336 312 183 315 324 173 288 339 239 272 207 220 255 289 244 305 226 230 254 284 206 328 270 291 329 178 225 258 175 283 279 191 246 192 266 309 260 323 217 313 236 198 299 301 232 234 281 248 249 327 253 268 224 316 222 259 322 326 221 171 292 180 194 304 340 274 235 332 257 174 227 282 333 200 330 190 233 250 213 231 247 280 294 278 276 187 243 269 216 228 285 176 204 287 293 215 251 303 320 197 205 238 172 196 271 318 179 286 306 201 211 252 308 338 188 297 310 319 186 223 290 195 265 184 237 177 298 325 300 311 208 229 181 264 295 321 273 296 241 261]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 replication_type:capi filter_expression: active:true target_nozzle_per_node:2 log_level:Info source_nozzle_per_node:2 stats_interval:1000]
CheckpointManager 2017-04-25T13:46:51.280-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962...
AsyncListener 2017-04-25T13:46:51.297-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:46:51.298-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:46:51.298-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:46:51.298-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:46:51.298-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:46:51.298-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:46:51.298-05:00 [INFO] Checkpointing starts in 1803.251 sec
PipelineRuntimeCtx 2017-04-25T13:46:51.298-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:46:51.298-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:46:51.298-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:46:51.298-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:46:51.300-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:46:51.300-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:46:51.300-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:46:51.300-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:46:51.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:46:51.344-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:46:51.345-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:46:51.345-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:46:51.358-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:46:51.358-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:46:51.362-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:46:51.362-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:46:51.362-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:46:51.365-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:46:51.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:46:51.382-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:46:51.405-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:46:51.405-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:51.405-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:46:51.405-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:46:51.405-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:46:51.406-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:46:51.435-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:46:51.435-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:46:51.435-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:46:51.461-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:46:51.461-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:46:51.576-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:46:51.635-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:46:51.635-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:46:51.635-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:46:51.635-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:46:51.635-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:46:51.635-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:46:51.635-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962 is started----------
PipelineManager 2017-04-25T13:46:51.635-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:46:51.635-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:46:51.635-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:46:51.812-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:46:51.812-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:46:51.812-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:46:51.812-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:46:51 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:46:51.952-05:00 [INFO] Send batch count=32
2017/04/25 13:46:51 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:46:51 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:46:51.999-05:00 [INFO] Send batch count=237
CapiNozzle 2017-04-25T13:46:52.007-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:52.009-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:46:52 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:46:52.065-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:46:52 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:46:52.085-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:52.086-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:52.120-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:46:52 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 109 successful
CapiNozzle 2017-04-25T13:46:52.170-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:52.205-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
2017/04/25 13:46:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:46:52.280-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:52.301-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
2017/04/25 13:46:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 324 successful
StatisticsManager 2017-04-25T13:46:52.365-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:46:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 234 successful
CapiNozzle 2017-04-25T13:46:52.487-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:52.521-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:46:52.521-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:46:52.521-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:46:52.521-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
ReplicationManager 2017-04-25T13:46:52.521-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]
PipelineManager 2017-04-25T13:46:52.529-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:46:52.529-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:46:52.535-05:00 [INFO] err_list=[{"time":"2017-04-25T13:46:52.535254692-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]
PipelineManager 2017-04-25T13:46:52.535-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:46:52 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 146 successful
CapiNozzle 2017-04-25T13:46:52.542-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:46:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 148 successful
CapiNozzle 2017-04-25T13:46:52.649-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:52.649-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:52.675-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
2017/04/25 13:46:52 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:46:52 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 89 successful
CapiNozzle 2017-04-25T13:46:53.132-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:46:53 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 293 successful
CapiNozzle 2017-04-25T13:46:53.154-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:46:53 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 99 successful
CheckpointManager 2017-04-25T13:46:53.225-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
CheckpointManager 2017-04-25T13:46:53.225-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:46:53 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:46:53 UPR_STREAMREQ for vb 297 successful
StatisticsManager 2017-04-25T13:46:53.312-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:46:53.312-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962 {"Errors": "[{\"time\":\"2017-04-25T13:46:52.535254692-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 113, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 14632, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 113, "set_filtered": 0, "set_received_from_dcp": 14632, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]", "Status": "Pending"}

DcpNozzle 2017-04-25T13:46:53.312-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:46:53.315-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:46:53.332-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
DcpNozzle 2017-04-25T13:46:53.344-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:46:54.063-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:46:54.063-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:46:54.063-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:46:54.063-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:54.063-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:54.153-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:54.155-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:54.179-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:46:54.282-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:46:54.306-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:46:54.895-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:46:54.927-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:46:55.066-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:46:55.066-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:46:55.066-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:46:55.066-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:55.066-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:55.162-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:55.162-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:55.179-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:46:55.641-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:46:55.667-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:46:55.870-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:55.900-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:46:56.442-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:46:56.442-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:46:56.442-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:46:56.442-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:56.442-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:56.508-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:56.509-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:56.539-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:46:56.637-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:56.656-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:46:57.217-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:57.249-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:46:57.399-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:46:57.399-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:46:57.399-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:46:57.399-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:57.399-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:57.467-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:57.467-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:57.486-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:46:58.155-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:58.169-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:46:58.312-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:46:58.330-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:46:59.001-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:46:59.001-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:46:59.001-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:46:59.001-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:46:59.001-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:46:59.123-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:46:59.123-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:46:59.134-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:46:59.360-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:46:59.371-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:47:00.805-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:47:00.816-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:47:01.091-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:47:01.091-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:47:01.091-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:47:01.091-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:01.091-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:01.175-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:01.175-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:01.186-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
GenericSupervisor 2017-04-25T13:47:01.299-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:47:01.299-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:47:01.299-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:47:01.633-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:47:01.710-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:47:01.721-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
PipelineManager 2017-04-25T13:47:01.742-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:52.535254692-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries]}
]
CapiNozzle 2017-04-25T13:47:02.003-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:02.016-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
PipelineManager 2017-04-25T13:47:02.535-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries 
PipelineManager 2017-04-25T13:47:02.535-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:47:02.535-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:47:02.535-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
PipelineRuntimeCtx 2017-04-25T13:47:02.535-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:47:02.535-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:47:02.535-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:47:02.535-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:47:02.535-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:02.535-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:47:02.536-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:02.535938762 -0500 CDT
GenericSupervisor 2017-04-25T13:47:02.536-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:02.536-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:47:02.536-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:02.536-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:47:02.536-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:02.536-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:47:02.536-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-189758962 has been stopped
GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-189758962 is stopped
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
PipelineManager 2017-04-25T13:47:02.536-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
PipelineManager 2017-04-25T13:47:02.536-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:46:52.535254692-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 101 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-189758962
PipelineManager 2017-04-25T13:47:02.536-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:02.536-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:47:02.536-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:47:02.536-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:02.536-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:47:02.536-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:47:02.537-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:47:02.537-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:02.537-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:47:02.537-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:47:02.537-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:47:02.537-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:47:02.537-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:02.537110937 -0500 CDT
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:47:02.537-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:47:02.537-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:02.537438535 -0500 CDT
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:47:02.537-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:02.537-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:47:02.538-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:47:02.538-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:47:02.538-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:47:02.538-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:47:02.538-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:47:02.538-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:02.538-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:47:02.538-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:02.539-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:47:02.539-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:02.539-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:02.539451608 -0500 CDT
DcpNozzle 2017-04-25T13:47:02.539-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:47:02.539-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:02.539-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:47:02.540-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:02.540-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:02.540360965 -0500 CDT
DcpNozzle 2017-04-25T13:47:02.540-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:47:02.540-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:02.540-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:47:02 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:47:02 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:47:02 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:47:02 Stream Closed for vb 173 StreamEnd simulated
XDCRFactory 2017-04-25T13:47:02.650-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:02.651-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:47:02.651-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:47:02.708-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:02.709-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:47:02.709-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:02.709-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:47:02.709-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:02.709-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:47:02.709-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:47:02.710-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:47:02.710-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
GenericSupervisor 2017-04-25T13:47:02.712-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:47:02.712-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
GenericSupervisor 2017-04-25T13:47:02.712-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:02.712-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:02.712-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:02.712-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:02.712-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:47:02.712-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
CheckpointManager 2017-04-25T13:47:02.712-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
CapiUtils 2017-04-25T13:47:02.793-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:02.793-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:02.793-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:47:02.794-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
StatisticsManager 2017-04-25T13:47:02.794-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:47:02.794-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
XDCRFactory 2017-04-25T13:47:02.794-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:47:02.795-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427 is constructed, start it
GenericPipeline 2017-04-25T13:47:02.796-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[149 153 158 72 62 79 128 170 30 61 106 8 122 129 107 22 37 57 164 169 58 116 23 59 65 66 93 95 101 124 150 151 162 108 159 21 28 133 13 14 71 47 113 117 123 12 112 20 16 138 3 24 111 119 45 73 82 110 142 67 105 160 92 125 51 84 155 29 32 39 100 127 15 99 48 90 126 166 7 10 17 25 136 143 4 38 49 55 77 130 1 56 132 146 54 109 11 52 64 70 6 134 144 46 114 135 44 88 121 137 53 85 91 33 78 96 2 102 140 148 157 50 86 141 5 97 145 26 41 80 81 115 40 89 165 43 34 168 154 19 68 74 103 167 76 0 94 161 27 63 87 156 60 120 147 18 36 83 118 152 104 163 75 69 9 35 42 98 31 131 139] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[227 297 258 174 236 241 280 251 261 313 334 178 275 263 184 264 175 223 318 301 199 266 331 183 187 202 271 190 214 221 322 311 205 259 262 274 252 172 176 211 326 337 230 232 235 304 180 197 243 246 247 218 177 193 207 265 188 215 273 294 333 196 267 319 186 325 255 269 179 244 257 307 233 279 339 283 195 289 242 228 312 314 192 240 306 220 222 229 253 285 231 239 288 291 206 305 330 234 324 209 208 268 296 321 200 248 272 198 276 332 201 217 281 308 260 338 303 323 237 286 203 210 245 315 219 270 173 302 340 171 182 189 278 299 295 256 336 320 249 290 292 293 282 316 328 329 213 216 284 181 194 250 225 335 185 226 277 212 298 327 204 287 309 317 191 300 238 254 310 224]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[72 62 79 128 170 30 61 106 8 122 129 107 22 37 57 164 169 58 101 116 23 59 65 66 93 95 124 150 151 162 108 159 21 28 133 13 14 71 47 113 117 123 12 112 20 16 138 3 24 111 119 45 73 82 110 142 67 105 160 92 125 51 84 155 29 32 39 100 127 15 99 48 90 126 166 7 10 17 25 136 143 4 38 49 55 77 130 1 56 132 146 54 109 11 52 64 70 6 134 144 46 114 135 44 88 121 137 53 85 91 33 78 96 2 102 140 148 157 50 86 141 5 97 145 26 41 80 81 115 40 89 165 43 34 168 154 19 68 74 103 167 76 0 94 161 27 63 87 156 60 120 147 18 36 83 118 152 104 163 75 69 9 35 42 98 31 131 139 149 153 158] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[241 258 174 236 280 251 261 313 334 178 275 263 184 264 175 223 318 266 301 199 331 183 187 202 271 190 214 221 322 274 311 205 259 262 252 172 176 211 326 337 230 232 235 247 304 180 197 243 246 207 218 177 193 215 265 188 273 294 333 196 267 319 186 325 255 269 179 244 257 307 339 233 279 195 283 289 228 242 312 306 314 192 240 285 220 222 229 253 231 239 288 291 206 305 330 234 324 209 321 208 268 296 200 248 272 198 276 308 332 201 217 281 260 338 323 303 237 286 315 203 210 245 219 270 173 302 278 340 171 182 189 299 295 256 336 320 249 290 292 293 329 282 316 328 284 213 216 181 194 250 335 225 185 226 277 212 298 327 204 287 309 317 191 300 238 254 310 224 227 297]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info replication_type:capi worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 filter_expression: active:true checkpoint_interval:1800 doc_batch_size_kb:2048 stats_interval:1000]
CheckpointManager 2017-04-25T13:47:02.796-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427...
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:02.830-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:47:02.830-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:02.830-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:47:02.830-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:47:02.830-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:47:02.830-05:00 [INFO] Checkpointing starts in 1800.655 sec
PipelineRuntimeCtx 2017-04-25T13:47:02.830-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:47:02.830-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:47:02.831-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:47:02.831-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:47:02.832-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:47:02.832-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:47:02.832-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:47:02.833-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:47:02.835-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:47:02.861-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:47:02.861-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:47:02.861-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:47:02.879-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:02.879-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:02.880-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:47:02.889-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:47:02.889-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:47:02.895-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:47:02.895-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:47:02.899-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:47:02.927-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:47:02.927-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:02.927-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:47:02.928-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:47:02.928-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:47:02.928-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:47:02.951-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:47:02.951-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:47:02.951-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:47:02.970-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:47:02.970-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:47:02.995-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:47:03.073-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:47:03.073-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:03.073-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:47:03.073-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:47:03.073-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:47:03.073-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:47:03.073-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427 is started----------
PipelineManager 2017-04-25T13:47:03.073-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:47:03.073-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:47:03.073-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:47:03.166-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:47:03.167-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:47:03.167-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:47:03.167-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:47:03 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:47:03.243-05:00 [INFO] Send batch count=102
2017/04/25 13:47:03 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:47:03.262-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:03.263-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:47:03 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:47:03.273-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:47:03 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 310 successful
CapiNozzle 2017-04-25T13:47:03.346-05:00 [INFO] Send batch count=57
2017/04/25 13:47:03 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 114 successful
CapiNozzle 2017-04-25T13:47:03.372-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:03.372-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:47:03 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 317 successful
CapiNozzle 2017-04-25T13:47:03.406-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:47:03.473-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:47:03 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 124 successful
CapiNozzle 2017-04-25T13:47:03.493-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:47:03.506-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:47:03 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 229 successful
CapiNozzle 2017-04-25T13:47:03.522-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
2017/04/25 13:47:03 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:47:03 UPR_STREAMREQ for vb 48 successful
CapiNozzle 2017-04-25T13:47:05.226-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:05.254-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:05.257-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:47:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 249 successful
StatisticsManager 2017-04-25T13:47:05.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:47:05.308-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:47:05 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 171 successful
CapiNozzle 2017-04-25T13:47:05.626-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:05.626-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:47:05.626-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:47:05.626-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
ReplicationManager 2017-04-25T13:47:05.626-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
PipelineManager 2017-04-25T13:47:05.643-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:47:05.643-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:47:05.646-05:00 [INFO] err_list=[{"time":"2017-04-25T13:47:05.646172009-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]
PipelineManager 2017-04-25T13:47:05.651-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:47:05 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 87 successful
CapiNozzle 2017-04-25T13:47:05.796-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:05.797-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:47:05 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 190 successful
CapiNozzle 2017-04-25T13:47:05.816-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:47:05.835-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:05.835-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:47:05.835-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:47:05.835-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:05.835-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:47:05.841-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:47:05.842-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427 {"Errors": "[{\"time\":\"2017-04-25T13:47:05.646172009-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 64, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 14381, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 64, "set_filtered": 0, "set_received_from_dcp": 14381, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]", "Status": "Pending"}

2017/04/25 13:47:05 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:47:05 UPR_STREAMREQ for vb 92 successful
CapiNozzle 2017-04-25T13:47:05.953-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:05.953-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:05.988-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:47:06 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 292 successful
DcpNozzle 2017-04-25T13:47:06.134-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CheckpointManager 2017-04-25T13:47:06.153-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
CheckpointManager 2017-04-25T13:47:06.153-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:47:06 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:47:06 UPR_STREAMREQ for vb 296 successful
DcpNozzle 2017-04-25T13:47:06.280-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:47:06.601-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:06.633-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:47:06.793-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:06.811-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:47:07.416-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:07.443-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:47:07.625-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:07.639-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:47:08.180-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:08.180-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:47:08.180-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:47:08.180-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:08.180-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:08.306-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:08.306-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:08.321-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:47:08.413-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:08.414-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:47:08.414-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:47:08.414-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:08.414-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:08.482-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:08.482-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:08.500-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:47:09.026-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:09.045-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:47:09.203-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:09.217-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:47:09.696-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:09.711-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:47:09.786-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:09.808-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:47:10.189-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:10.189-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:47:10.189-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:47:10.189-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:10.189-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:10.254-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:10.254-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:10.270-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:47:10.498-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:10.498-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:47:10.498-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:47:10.498-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:10.498-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:10.592-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:10.592-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:10.608-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:47:10.970-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:47:10.993-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:47:11.547-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:11.563-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:47:12.500-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:47:12.511-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
GenericSupervisor 2017-04-25T13:47:12.831-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:47:12.831-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:47:12.831-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:47:13.124-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:47:13.250-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:13.265-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:47:13.872-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:47:13.872-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:47:13.872-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:47:13.872-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:13.872-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:13.962-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:13.962-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:13.973-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:47:14.365-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:14.365-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:47:14.365-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:47:14.365-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:14.365-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:14.458-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:14.458-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:14.469-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:47:14.986-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:47:14.997-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
PipelineManager 2017-04-25T13:47:15.651-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries 
PipelineManager 2017-04-25T13:47:15.651-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:47:15.651-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:47:15.651-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
PipelineRuntimeCtx 2017-04-25T13:47:15.651-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:47:15.651-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:15.651-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:47:15.651-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:15.651665571 -0500 CDT
GenericSupervisor 2017-04-25T13:47:15.651-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:15.651-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:47:15.651-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:15.651-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:47:15.651-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:15.651-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:47:15.651-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:47:15.652-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:47:15.652-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:47:15.652-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-709752427 has been stopped
GenericPipeline 2017-04-25T13:47:15.652-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-709752427 is stopped
PipelineManager 2017-04-25T13:47:15.652-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:47:15.652-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:47:05.646172009-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:47:15.652-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:47:15.652-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
GenericPipeline 2017-04-25T13:47:15.652-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
GenericPipeline 2017-04-25T13:47:15.652-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
GenericPipeline 2017-04-25T13:47:15.652-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-709752427
AsyncListener 2017-04-25T13:47:15.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:15.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:15.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:47:15.653-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:47:15.653-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:47:15.653-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:47:15.653-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:15.653-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:47:15.653-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:47:15.653-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:47:15.653-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:47:15.653-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:15.654-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:47:15.654-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:15.654-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:47:15.654-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:15.653654608 -0500 CDT
DcpNozzle 2017-04-25T13:47:15.654-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:47:15.654-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:15.653997232 -0500 CDT
DcpNozzle 2017-04-25T13:47:15.654-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:47:15.655-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:47:15.655-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:47:15.655-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:47:15.655-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:47:15.655-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:47:15.655-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:15.656-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:15.656-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:47:15.656-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:15.656-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:47:15.660-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:15.660-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:15.660-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:15.660323737 -0500 CDT
DcpNozzle 2017-04-25T13:47:15.660-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:15.660341057 -0500 CDT
DcpNozzle 2017-04-25T13:47:15.660-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:47:15.660-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:47:15.660-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:15.660-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:47:15.660-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:15.660-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:47:15 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:47:15 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:47:15 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:47:15 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:47:15.773-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:15.774-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:47:15.774-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:47:15.894-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:15.895-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:47:15.896-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:15.896-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:47:15.896-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:15.896-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:47:15.896-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:47:15.897-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:47:15.897-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
GenericSupervisor 2017-04-25T13:47:15.900-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:47:15.900-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
GenericSupervisor 2017-04-25T13:47:15.900-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:15.900-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:15.900-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:15.900-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:15.900-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:47:15.900-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
CheckpointManager 2017-04-25T13:47:15.900-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
CapiUtils 2017-04-25T13:47:15.972-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:15.972-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:15.972-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:47:15.973-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
StatisticsManager 2017-04-25T13:47:15.973-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:47:15.973-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
XDCRFactory 2017-04-25T13:47:15.974-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:47:15.974-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144 is constructed, start it
GenericPipeline 2017-04-25T13:47:15.975-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[28 45 61 95 14 105 137 68 18 150 11 119 25 21 69 143 144 163 8 167 141 41 155 16 135 139 140 19 109 108 110 88 99 154 89 44 60 29 75 37 63 77 106 112 121 127 170 7 126 86 166 133 9 0 124 146 23 30 87 93 12 59 84 147 54 157 22 67 79 42 33 101 142 3 92 72 53 10 103 5 114 51 136 15 83 35 39 56 117 6 78 111 113 115 120 151 153 36 158 102 64 138 97 149 27 62 98 128 169 24 73 90 125 132 165 70 159 130 34 71 162 2 82 31 74 94 107 20 104 161 65 160 118 38 57 81 17 76 85 168 58 48 116 129 46 100 164 47 80 96 122 131 66 52 123 1 40 49 55 148 152 4 32 91 13 134 156 50 43 145 26] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[251 270 281 287 325 282 324 219 210 316 340 171 207 253 276 313 176 197 232 236 299 315 243 296 307 184 329 275 323 327 188 336 186 266 300 302 311 338 330 225 221 245 249 202 255 294 339 189 303 314 179 319 234 203 204 213 183 172 318 208 190 217 211 268 284 290 247 187 191 214 250 265 286 305 285 292 239 283 199 259 267 278 291 295 297 231 261 274 304 332 195 196 220 246 252 224 233 269 192 194 337 201 280 288 254 177 321 206 227 257 200 237 264 320 226 331 277 263 312 223 293 306 216 241 301 193 209 262 180 205 173 174 178 271 273 238 248 289 322 326 182 215 242 279 175 260 317 198 222 333 310 218 228 256 258 185 212 235 309 230 181 240 298 308 328 272 334 229 244 335]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[68 105 137 11 18 150 25 119 143 144 163 8 21 69 141 167 16 41 155 140 19 135 139 108 109 88 110 89 99 154 29 44 60 37 75 106 112 121 127 170 7 63 77 86 126 133 166 0 9 124 146 87 93 12 23 30 59 54 84 147 22 157 42 67 79 142 3 33 101 72 92 10 53 5 103 114 15 51 136 35 83 117 6 39 56 113 115 120 151 153 36 78 111 158 64 102 138 27 97 149 128 169 24 62 98 125 132 165 70 73 90 130 159 162 2 34 71 82 94 107 20 31 74 65 104 161 118 160 81 17 38 57 76 58 85 168 129 46 48 116 47 100 164 80 131 66 96 122 1 52 123 55 148 152 4 40 49 13 32 91 50 134 156 43 26 145 61 95 14 28 45] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[219 210 316 340 207 253 276 313 171 176 197 232 236 299 315 243 296 307 329 184 323 327 275 188 336 186 266 300 302 311 330 338 225 245 249 221 294 339 189 202 255 303 314 319 179 234 213 183 203 204 172 318 208 190 217 211 268 284 290 247 187 191 214 250 265 285 286 305 292 283 239 199 267 278 291 295 259 297 231 261 274 304 332 196 220 246 252 195 224 233 269 337 192 194 201 254 280 288 177 321 257 200 206 227 237 264 320 226 331 277 312 263 223 293 306 301 216 241 193 262 209 180 205 178 271 273 173 174 238 248 289 322 326 182 215 242 279 175 260 317 333 198 222 310 218 228 256 258 185 212 230 235 309 181 240 298 308 328 272 334 335 229 244 251 270 281 282 287 325 324]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[failure_restart_interval:10 filter_expression: active:true checkpoint_interval:1800 doc_batch_size_kb:2048 target_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:47:15.975-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144...
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:15.997-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:47:15.997-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:15.997-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:47:15.997-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:47:15.997-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:47:15.997-05:00 [INFO] Checkpointing starts in 1804.565 sec
PipelineRuntimeCtx 2017-04-25T13:47:15.997-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:47:15.997-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:47:15.997-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:47:15.997-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:47:15.998-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:47:15.998-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:47:15.998-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:47:15.998-05:00 [INFO] Service TopologyChangeDetectSvc has been started
CapiNozzle 2017-04-25T13:47:15.999-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
ToplogyChangeDetector 2017-04-25T13:47:16.033-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:47:16.043-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:47:16.043-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:47:16.043-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:47:16.059-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:47:16.059-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:47:16.066-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:47:16.070-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:16.070-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:16.071-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:47:16.077-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:47:16.079-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:47:16.079-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:16.079-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:47:16.079-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:47:16.079-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:47:16.080-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:47:16.117-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:47:16.118-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:47:16.118-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:47:16.134-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:47:16.134-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:47:16.207-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:47:16.313-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:47:16.313-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:16.313-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:47:16.334-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:47:16.334-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:47:16.335-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:47:16.335-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144 is started----------
PipelineManager 2017-04-25T13:47:16.335-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:47:16.335-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:47:16.335-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:47:16.408-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:47:16.408-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:47:16.408-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:47:16.409-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:47:16 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:47:16.505-05:00 [INFO] Send batch count=100
2017/04/25 13:47:16 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:47:16.524-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:16.524-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:16.530-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:47:16.533-05:00 [INFO] Send batch count=56
CapiNozzle 2017-04-25T13:47:16.538-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:47:16 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:47:16.582-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:16.582-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:47:16 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:47:16.613-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:47:16 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 11 successful
PipelineManager 2017-04-25T13:47:16.691-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:47:16 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 119 successful
CapiNozzle 2017-04-25T13:47:16.792-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:47:16 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 214 successful
CapiNozzle 2017-04-25T13:47:16.817-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:47:16 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 122 successful
CapiNozzle 2017-04-25T13:47:16.878-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:16.902-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
2017/04/25 13:47:16 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:47:16 UPR_STREAMREQ for vb 230 successful
CapiNozzle 2017-04-25T13:47:17.022-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:47:17.056-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:47:17.058-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:47:17.065-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:47:17 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 141 successful
CapiNozzle 2017-04-25T13:47:17.099-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:47:17 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 58 successful
CapiNozzle 2017-04-25T13:47:17.264-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:17.264-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:47:17.264-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:47:17.264-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
ReplicationManager 2017-04-25T13:47:17.264-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]
PipelineManager 2017-04-25T13:47:17.283-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:47:17.283-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:47:17.286-05:00 [INFO] err_list=[{"time":"2017-04-25T13:47:17.286573393-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]
PipelineManager 2017-04-25T13:47:17.286-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:47:17 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 166 successful
CapiNozzle 2017-04-25T13:47:17.374-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:17.374-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:17.386-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:47:17 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 265 successful
CapiNozzle 2017-04-25T13:47:17.427-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:17.427-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:47:17.427-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:47:17.427-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:17.427-05:00 [INFO] Send batch count=500
2017/04/25 13:47:17 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 78 successful
CapiNozzle 2017-04-25T13:47:17.505-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:17.506-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:47:17 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 179 successful
CapiNozzle 2017-04-25T13:47:17.523-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:47:17 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 196 successful
CheckpointManager 2017-04-25T13:47:17.732-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
CheckpointManager 2017-04-25T13:47:17.732-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:47:17 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:47:17 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:47:17.852-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:47:17.884-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:47:17.958-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:17.976-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
StatisticsManager 2017-04-25T13:47:18.008-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:47:18.008-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144 {"Errors": "[{\"time\":\"2017-04-25T13:47:17.286573393-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 63, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 21059, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 63, "set_filtered": 0, "set_received_from_dcp": 21059, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:47:18.180-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:18.192-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:47:18.649-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:18.663-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:47:19.021-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:19.043-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:47:19.426-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:19.426-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:47:19.426-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:47:19.426-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:19.426-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:19.541-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:19.541-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:19.574-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:47:19.903-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:19.903-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:47:19.903-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:47:19.903-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:19.903-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:20.066-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:20.066-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:20.092-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:47:20.365-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:47:20.382-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:47:20.648-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:20.668-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:47:21.089-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:47:21.117-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:47:21.188-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:21.202-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:47:21.810-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:47:21.810-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:47:21.810-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:47:21.810-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:21.810-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:21.893-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:21.901-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:21.914-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:47:21.960-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:47:21.960-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:47:21.960-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:47:21.960-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:21.960-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:22.043-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:22.043-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:22.066-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:47:22.710-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:22.725-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:47:22.857-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:47:22.870-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
CapiNozzle 2017-04-25T13:47:24.011-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:24.021-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:47:24.482-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:47:24.492-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:47:25.296-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:47:25.296-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:47:25.296-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:47:25.296-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:25.296-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:25.391-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:25.391-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:25.405-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:47:25.762-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:47:25.762-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:47:25.762-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:47:25.762-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:25.762-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:25.848-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:25.848-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:25.859-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
GenericSupervisor 2017-04-25T13:47:25.998-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:47:25.998-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:47:25.998-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:47:26.233-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:47:26.725-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:47:26.736-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:47:26.907-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:47:26.918-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
PipelineManager 2017-04-25T13:47:27.286-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries 
PipelineManager 2017-04-25T13:47:27.287-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:47:27.287-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:47:27.287-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
PipelineRuntimeCtx 2017-04-25T13:47:27.287-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:47:27.287-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:27.287-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:47:27.287-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:27.287201506 -0500 CDT
GenericSupervisor 2017-04-25T13:47:27.287-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:27.287-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:47:27.287-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:27.287-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:47:27.287-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:27.287-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:47:27.287-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:47:27.287-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:47:27.287-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:47:27.287-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-896575144 has been stopped
GenericPipeline 2017-04-25T13:47:27.287-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-896575144 is stopped
PipelineManager 2017-04-25T13:47:27.287-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:47:27.288-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:47:17.286573393-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 302 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:47:27.288-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:47:27.288-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:47:27.288-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:47:27.288-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
GenericPipeline 2017-04-25T13:47:27.288-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-896575144
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:27.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:47:27.289-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:27.289-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:47:27.289-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:47:27.289-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:47:27.289-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:47:27.289-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:47:27.289-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:47:27.289-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:47:27.289-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:47:27.290-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:27.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:47:27.290-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:27.289619177 -0500 CDT
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:47:27.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:47:27.290-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:27.290076007 -0500 CDT
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:27.290-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:47:27.291-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:47:27.291-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:47:27.291-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:47:27.291-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:47:27.291-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:47:27.291-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:47:27.291-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:27.291-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:47:27.291-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:27.291-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:47:27.293-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:27.293-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:27.293497878 -0500 CDT
DcpNozzle 2017-04-25T13:47:27.293-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:47:27.293-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:27.293-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:47:27.293-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:27.293-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:27.293851702 -0500 CDT
DcpNozzle 2017-04-25T13:47:27.293-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:47:27.293-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:27.293-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:47:27 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:47:27 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:47:27.454-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:27.454-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:47:27.454-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:47:27.544-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:27.545-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:47:27.545-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:27.545-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:47:27.545-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:27.545-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:47:27.545-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:47:27.546-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:47:27.546-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
GenericSupervisor 2017-04-25T13:47:27.548-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:47:27.548-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
GenericSupervisor 2017-04-25T13:47:27.548-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:27.548-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:27.548-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:27.548-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:27.548-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:47:27.548-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
CheckpointManager 2017-04-25T13:47:27.548-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
CapiUtils 2017-04-25T13:47:27.652-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:27.652-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:27.652-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:47:27.653-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
StatisticsManager 2017-04-25T13:47:27.653-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:47:27.653-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
XDCRFactory 2017-04-25T13:47:27.654-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:47:27.654-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161 is constructed, start it
GenericPipeline 2017-04-25T13:47:27.655-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[8 96 34 145 21 90 107 153 130 135 64 94 111 143 16 133 144 10 15 28 52 106 113 59 89 155 40 140 162 166 6 50 54 105 4 138 30 31 86 95 100 152 157 29 76 26 36 159 148 108 109 127 165 22 116 147 63 82 84 170 37 121 156 85 5 38 81 104 137 158 65 91 110 129 14 146 32 43 53 132 161 23 35 57 125 134 3 62 98 154 24 92 102 112 78 39 18 9 66 83 126 11 71 120 80 139 19 46 55 118 142 20 79 101 7 88 122 128 167 47 61 87 164 151 160 115 42 69 163 41 56 141 149 27 67 60 68 114 45 2 49 58 70 72 136 169 13 74 124 131 1 44 123 33 99 103 150 25 51 77 97 0 12 17 48 73 75 93 119 117 168] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[285 189 208 237 309 325 326 267 310 315 224 207 241 251 258 271 275 300 334 335 283 293 261 284 311 312 200 256 297 183 204 216 223 230 211 332 209 175 318 197 225 274 292 254 295 338 219 329 182 218 263 270 282 246 190 196 266 298 176 210 213 247 317 337 202 234 244 319 198 214 305 181 288 235 260 193 206 236 339 173 192 199 233 279 286 287 185 203 290 322 336 179 257 242 278 171 248 302 323 331 340 222 280 227 212 250 294 205 243 255 281 306 327 191 249 276 299 316 320 187 217 239 273 301 268 231 328 291 308 324 240 289 304 252 269 272 330 178 184 321 221 303 172 188 307 186 194 195 253 262 296 201 215 229 238 245 259 174 177 277 313 333 228 265 180 226 264 220 232 314]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[152 157 29 76 26 36 159 148 108 109 127 165 22 116 147 63 82 84 170 37 121 156 85 5 38 81 104 137 158 65 91 110 129 14 146 32 43 53 132 161 23 35 57 125 134 3 62 98 154 24 92 102 112 78 39 18 9 66 83 126 11 71 120 80 139 19 46 55 118 142 20 79 101 7 88 122 128 167 47 61 87 164 151 160 115 42 69 163 41 56 141 149 27 67 60 68 114 45 2 49 58 70 72 136 169 13 74 124 131 1 44 123 33 99 103 150 25 51 77 97 0 12 17 48 73 75 93 119 117 168 8 96 34 145 21 90 107 153 130 135 64 94 111 143 16 133 144 10 15 28 52 106 113 59 89 155 40 140 162 166 6 50 54 105 4 138 30 31 86 95 100] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[175 318 197 225 274 292 254 295 338 219 329 182 218 263 270 282 246 190 196 266 298 176 210 213 247 317 337 202 234 244 319 198 214 305 181 288 235 260 193 206 236 339 173 192 199 233 279 286 287 185 203 290 322 336 179 257 242 278 171 248 302 323 331 340 222 280 227 212 250 294 205 243 255 281 306 327 191 249 276 299 316 320 187 217 239 273 301 268 231 328 291 308 324 240 289 304 252 269 272 330 178 184 321 221 303 172 188 307 186 194 195 253 262 296 201 215 229 238 245 259 174 177 277 313 333 228 265 180 226 264 220 232 314 285 189 208 237 309 325 326 267 310 315 224 207 241 251 258 271 275 300 334 335 283 293 261 284 311 312 200 256 297 183 204 216 223 230 211 332 209]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true worker_batch_size:500 source_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:47:27.655-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161...
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:27.662-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:47:27.662-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:27.662-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:47:27.662-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:47:27.662-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:47:27.662-05:00 [INFO] Checkpointing starts in 1803.701 sec
PipelineRuntimeCtx 2017-04-25T13:47:27.662-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:47:27.663-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:47:27.663-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:47:27.663-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:47:27.664-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:47:27.664-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:47:27.664-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:47:27.664-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:47:27.668-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:47:27.692-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:47:27.692-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:47:27.693-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:47:27.710-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:47:27.710-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:47:27.726-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiUtils 2017-04-25T13:47:27.729-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:27.729-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:27.729-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:47:27.734-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:47:27.751-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:47:27.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:47:27.753-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:27.753-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:47:27.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:47:27.753-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:47:27.753-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:47:27.781-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:47:27.782-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:47:27.782-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:47:27.803-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:47:27.803-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:47:27.933-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:47:27.933-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:27.933-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:47:27.933-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:47:27.933-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:47:27.933-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:47:27.933-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161 is started----------
PipelineManager 2017-04-25T13:47:27.934-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:47:27.934-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:47:27.934-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:47:27.937-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:47:28.179-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:47:28.179-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:47:28.179-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:47:28.179-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:47:28 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:47:28.330-05:00 [INFO] Send batch count=98
CapiNozzle 2017-04-25T13:47:28.330-05:00 [INFO] Send batch count=65
CapiNozzle 2017-04-25T13:47:28.352-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:28.352-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:28.353-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:28.353-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:47:28 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:47:28.385-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:47:28.390-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:47:28 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 29 successful
CapiNozzle 2017-04-25T13:47:28.736-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:47:28 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 329 successful
StatisticsManager 2017-04-25T13:47:28.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:47:28.764-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:47:28 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 130 successful
CapiNozzle 2017-04-25T13:47:28.816-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:28.830-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:47:28 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:47:28 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 54 successful
CapiNozzle 2017-04-25T13:47:29.099-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:47:29.139-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:47:29 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 280 successful
CapiNozzle 2017-04-25T13:47:29.246-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:29.265-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:47:29 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 190 successful
CheckpointManager 2017-04-25T13:47:29.462-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
CheckpointManager 2017-04-25T13:47:29.462-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:47:29 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:47:29 UPR_STREAMREQ for vb 94 successful
CapiNozzle 2017-04-25T13:47:29.486-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:47:29.486-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:47:29.486-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:47:29.486-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
ReplicationManager 2017-04-25T13:47:29.486-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
PipelineManager 2017-04-25T13:47:29.510-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:47:29.510-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:47:29.512-05:00 [INFO] err_list=[{"time":"2017-04-25T13:47:29.512877772-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:47:29.513-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
DcpNozzle 2017-04-25T13:47:29.539-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:47:29.542-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:29.542-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:47:29.542-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:47:29.542-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:29.542-05:00 [INFO] Send batch count=500
2017/04/25 13:47:29 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:47:29.588-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:29.589-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:29.627-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:29.627-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:29.632-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:47:29.660-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
StatisticsManager 2017-04-25T13:47:29.679-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:47:29.680-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161 {"Errors": "[{\"time\":\"2017-04-25T13:47:29.512877772-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 70, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 16428, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 70, "set_filtered": 0, "set_received_from_dcp": 16433, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]", "Status": "Pending"}

DcpNozzle 2017-04-25T13:47:29.686-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:47:30.275-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:30.309-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:47:30.443-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:30.477-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:47:31.215-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:31.231-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:47:31.402-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:31.421-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:47:34.133-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:47:34.261-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:35.714-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:47:35.714-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:47:35.714-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:35.714-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:35.731-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:47:35.731-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:47:35.731-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:35.731-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:47:35.807-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:47:29.512877772-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]}
]
ToplogyChangeDetector 2017-04-25T13:47:40.092-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:47:40.068-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:47:41.143-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:47:40.027-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:47:41.878-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:47:42.686-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:41.731-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:47:42.765-05:00 [INFO] ****************************
GenericPipeline 2017-04-25T13:47:42.779-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
PipelineRuntimeCtx 2017-04-25T13:47:42.796-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:47:43.358-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:43.380-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:43.673-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:47:43.713-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:47:43.715-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:43.664762145 -0500 CDT
GenericSupervisor 2017-04-25T13:47:45.864-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:47.107-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:47:47.319-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:47.471-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CheckpointManager 2017-04-25T13:47:47.292-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:47:47.599-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:47:47.619-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:47:47.751-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:47:47.931-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:47:47.931-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-545948161 has been stopped
CapiNozzle 2017-04-25T13:47:47.983-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:48.009-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericSupervisor 2017-04-25T13:47:47.880-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:47:48.059-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:47:47.922-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
GenericSupervisor 2017-04-25T13:47:48.116-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericPipeline 2017-04-25T13:47:48.128-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
GenericPipeline 2017-04-25T13:47:48.142-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
GenericPipeline 2017-04-25T13:47:48.146-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
GenericPipeline 2017-04-25T13:47:48.149-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-545948161
CapiNozzle 2017-04-25T13:47:48.151-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericPipeline 2017-04-25T13:47:48.161-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-545948161 is stopped
PipelineManager 2017-04-25T13:47:48.161-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:47:48.165-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:47:48.163-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:47:29.512877772-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

CapiNozzle 2017-04-25T13:47:48.165-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
PipelineManager 2017-04-25T13:47:48.167-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:47:48.169-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:47:48.169-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:47:48.194-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:47:48.194-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:47:48.194-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericSupervisor 2017-04-25T13:47:48.198-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
CapiNozzle 2017-04-25T13:47:48.201-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:48.169896731 -0500 CDT
CapiNozzle 2017-04-25T13:47:48.212-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:48.194953447 -0500 CDT
CapiNozzle 2017-04-25T13:47:48.216-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:47:48.223-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:48.238-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:48.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:48.239-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:48.241-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:48.242-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:48.243-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:48.243-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:48.243-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:48.243-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:47:48.245-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:47:48.250-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:48.250-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:47:48.253-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.253-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.253-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.253-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.255-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:47:48.252-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:47:48.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
DcpNozzle 2017-04-25T13:47:48.180-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:47:48.294-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:47:48.288-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:48.301-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:48.207-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.312-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:47:48.314-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:47:48.314-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:47:48.331-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:47:48.332-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:47:48.333-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:47:48.375-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:47:48.399-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
DcpNozzle 2017-04-25T13:47:48.433-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:48.433-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:48.433953973 -0500 CDT
DcpNozzle 2017-04-25T13:47:48.434-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:47:48.457-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:48.466-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:47:48.479-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:47:48.479-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:47:48.479-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:48.479-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
CapiNozzle 2017-04-25T13:47:48.556-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:47:48.556-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:47:48.556-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:48.556-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:47:48.577-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:47:48.577-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:47:48.577671054 -0500 CDT
DcpNozzle 2017-04-25T13:47:48.577-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:47:48.577-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:47:48.591-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericSupervisor 2017-04-25T13:47:48.668-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
ReplicationManager 2017-04-25T13:47:49.160-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
XDCRFactory 2017-04-25T13:47:49.211-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:49.282-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:47:49.283-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:47:49.639-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:47:49.647-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:47:49.650-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:49.650-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:47:49.650-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:47:49.650-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:47:49.650-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:47:49.651-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:47:49.660-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
GenericSupervisor 2017-04-25T13:47:49.663-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:47:49.663-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
GenericSupervisor 2017-04-25T13:47:49.664-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:49.664-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:49.664-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:49.664-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:47:49.664-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:47:49.669-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
CheckpointManager 2017-04-25T13:47:49.669-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
CapiUtils 2017-04-25T13:47:49.787-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:49.791-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:49.792-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:47:49.795-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
StatisticsManager 2017-04-25T13:47:49.803-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:47:49.804-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
XDCRFactory 2017-04-25T13:47:49.805-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:47:49.805-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344 is constructed, start it
GenericPipeline 2017-04-25T13:47:49.806-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[223 172 311 231 236 251 306 333 255 183 208 218 278 285 338 266 267 327 216 227 264 175 277 244 282 318 328 228 302 187 210 230 260 330 202 213 248 287 206 258 297 314 326 191 247 305 312 329 319 238 240 337 173 235 207 323 214 279 188 203 233 242 275 321 211 280 300 307 194 221 197 291 336 288 222 198 215 290 315 270 303 212 209 195 204 252 262 335 310 334 174 176 182 268 276 246 259 239 253 331 184 224 263 271 293 201 237 219 250 272 257 186 232 325 245 317 225 185 192 261 286 289 298 340 254 281 177 273 301 332 220 171 180 295 309 269 178 179 234 189 217 299 304 316 283 200 229 249 320 339 308 226 241 205 256 284 296 190 313 324 274 243 265 181 199 322 292 196 193 294] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[5 49 111 116 140 29 30 53 16 120 168 25 150 20 65 93 87 33 105 170 0 95 158 162 106 127 142 148 24 46 122 48 70 4 42 73 88 146 147 159 67 94 132 103 126 129 97 166 7 15 71 119 141 38 40 58 89 90 100 68 9 47 117 143 110 155 104 118 12 32 52 61 136 59 131 64 79 85 108 35 36 39 22 66 139 41 160 1 145 84 34 54 55 76 107 165 18 44 50 91 98 11 14 56 123 17 96 75 92 74 78 109 157 167 57 69 77 115 138 114 128 27 83 130 6 101 137 144 125 164 26 51 80 86 13 151 37 72 81 21 169 113 121 135 28 45 82 62 133 134 2 8 31 23 152 63 154 10 153 112 3 163 19 43 124 102 149 161 60 99 156]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[140 29 30 53 111 116 16 120 168 25 150 20 65 93 87 33 105 170 0 95 158 106 127 142 162 24 46 122 148 48 70 4 42 73 88 146 147 159 67 94 103 126 129 132 7 15 71 97 166 119 38 40 58 141 89 90 100 68 9 47 117 143 110 155 12 32 52 104 118 61 136 59 131 64 79 85 35 36 39 108 22 66 139 41 160 1 145 34 54 55 84 165 18 44 50 76 107 11 14 56 91 98 123 17 96 75 92 74 78 109 57 69 77 157 167 115 138 114 128 27 83 130 6 101 137 144 26 51 80 125 164 86 13 151 37 72 81 21 169 135 28 45 82 113 121 134 2 8 31 62 133 23 152 63 154 10 153 112 3 19 43 124 163 102 149 60 99 156 161 5 49] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[311 251 231 236 255 306 333 218 278 285 183 208 338 266 267 327 227 264 216 175 277 282 318 328 244 187 228 302 260 330 210 230 202 287 206 213 248 314 326 191 258 297 305 312 329 247 319 337 173 235 238 240 323 207 188 203 214 279 275 321 211 233 242 280 300 194 221 307 197 291 336 222 288 198 215 290 315 212 270 303 209 252 262 335 195 204 334 310 182 268 276 174 176 246 259 253 331 239 184 224 263 271 293 201 237 219 250 257 272 186 232 325 225 245 317 261 286 289 185 192 298 340 177 254 281 332 220 273 301 171 180 295 309 178 179 269 189 234 304 316 217 299 283 200 229 249 320 339 308 226 241 284 296 190 205 256 274 313 324 243 265 181 199 322 196 292 294 193 172 223]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi active:true checkpoint_interval:1800 filter_expression: worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:47:49.809-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344...
PipelineManager 2017-04-25T13:47:49.827-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:47:29.512877772-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is constructed}
]
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:47:49.864-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:47:49.865-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:47:49.865-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:47:49.865-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:47:49.879-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:47:49.866-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:47:49.892-05:00 [INFO] Checkpointing starts in 1804.371 sec
PipelineRuntimeCtx 2017-04-25T13:47:49.892-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:47:49.892-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:47:49.898-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:47:49.901-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:47:49.902-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:47:49.902-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:47:49.904-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:47:49.927-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:47:49.979-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:47:49.979-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:47:49.979-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:47:50.001-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:47:50.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:47:50.005-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:47:50.005-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:47:50.006-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:47:50.020-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:47:50.024-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:47:50.067-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:47:50.067-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
CheckpointManager 2017-04-25T13:47:50.184-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:47:50.428-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:47:50.428-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:50.428-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
CapiNozzle 2017-04-25T13:47:50.471-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
DcpNozzle 2017-04-25T13:47:50.462-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:47:50.471-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:47:50.493-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:47:50.493-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:47:50.493-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:47:50.514-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:47:50.514-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:47:51.063-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:47:51.734-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:47:51.734-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:47:51.734-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
CheckpointManager 2017-04-25T13:47:51.739-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:47:51.741-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:47:51.741-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:47:51.741-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:47:51.741-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344 is started----------
PipelineManager 2017-04-25T13:47:51.741-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:47:51.741-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:47:51.742-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:47:51.958-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:47:52.107-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:47:52.116-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:47:52.117-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:47:52.117-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:47:52 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 114 successful
CapiNozzle 2017-04-25T13:47:52.671-05:00 [INFO] Send batch count=98
2017/04/25 13:47:52 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 121 successful
CapiNozzle 2017-04-25T13:47:52.832-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:47:52.836-05:00 [INFO] Send batch count=119
2017/04/25 13:47:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:47:52 UPR_STREAMREQ for vb 150 successful
StatisticsManager 2017-04-25T13:47:53.028-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
2017/04/25 13:47:53 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 256 successful
CapiNozzle 2017-04-25T13:47:53.075-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:47:53 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 154 successful
CapiNozzle 2017-04-25T13:47:53.105-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:47:53 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 91 successful
CheckpointManager 2017-04-25T13:47:53.508-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
CheckpointManager 2017-04-25T13:47:53.508-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:47:53.537-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:47:53 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 195 successful
CapiNozzle 2017-04-25T13:47:53.549-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:47:53 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:47:53 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:47:53.642-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:47:53.675-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:47:53.905-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:53.932-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
StatisticsManager 2017-04-25T13:47:53.950-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:47:54.267-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:54.278-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:47:54.366-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:54.379-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:47:54.955-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
StatisticsManager 2017-04-25T13:47:54.968-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:47:54.971-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:47:55.044-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:55.044-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:47:55.044-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:47:55.044-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:47:55.044-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:47:55.327-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:47:55.327-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:47:55.330-05:00 [INFO] err_list=[{"time":"2017-04-25T13:47:55.330823037-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:47:55.333-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:47:55.428-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:55.428-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:55.465-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:47:55.523-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:55.523-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:47:55.523-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:47:55.523-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:47:55.537-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:47:55.592-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:47:55.592-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:47:55.603-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
StatisticsManager 2017-04-25T13:47:55.903-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:47:55.913-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344 {"Errors": "[{\"time\":\"2017-04-25T13:47:55.330823037-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 94, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 3, "docs_received_from_dcp": 32464, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 94, "set_filtered": 0, "set_received_from_dcp": 32464, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:47:55.966-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:55.978-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:47:56.738-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:47:57.151-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:47:58.434-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:47:59.142-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
GenericSupervisor 2017-04-25T13:48:00.024-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:48:00.076-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:48:00.089-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:48:01.030-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:48:01.044-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:48:01.207-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:48:01.211-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:48:01.211-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:48:01.211-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:01.223-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:01.381-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:01.405-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:01.432-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
ToplogyChangeDetector 2017-04-25T13:48:01.745-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:48:01.815-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:48:01.815-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:48:01.815-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:48:01.815-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:01.815-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:48:32.981-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:48:33.222-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:48:33.225-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:48:33.227-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
CapiNozzle 2017-04-25T13:48:33.230-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
PipelineRuntimeCtx 2017-04-25T13:48:33.332-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:48:33.334-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:48:33.513-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
GenericSupervisor 2017-04-25T13:48:33.543-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:48:33.549-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:48:33.456492406 -0500 CDT
GenericSupervisor 2017-04-25T13:48:33.556-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:48:33.556-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:48:33.558-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:48:33.560-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:48:33.561-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:48:33.561-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:48:33.565-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:48:33.565-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:48:33.567-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:48:33.567-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-650627344 has been stopped
GenericPipeline 2017-04-25T13:48:33.636-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-650627344 is stopped
PipelineManager 2017-04-25T13:48:33.636-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:48:33.641-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
GenericPipeline 2017-04-25T13:48:33.647-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
GenericPipeline 2017-04-25T13:48:33.647-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
GenericPipeline 2017-04-25T13:48:33.647-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-650627344
AsyncListener 2017-04-25T13:48:33.648-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:48:33.653-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:48:33.656-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:48:33.657-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
GenericPipeline 2017-04-25T13:48:33.667-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:48:33.668-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:48:33.670-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:48:33.670-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:48:33.670-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:48:33.670-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:48:33.676-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:48:33.676-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:48:33.676-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:48:33.676-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.677-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.679-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.680-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:48:33.680-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:48:33.680-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:48:33.670451491 -0500 CDT
DcpNozzle 2017-04-25T13:48:33.683-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:48:33.688-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:48:33.676987036 -0500 CDT
CapiNozzle 2017-04-25T13:48:33.688-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:48:33.688-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:48:33.688-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.692-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:48:33.692-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:48:33.693-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.694-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:48:33.694-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.694-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:48:33.694-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.694-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:48:33.694-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.695-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:48:33.695-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.695-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:48:33.695-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:48:33.695-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:48:33.661-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
panic: send on closed channel

goroutine 2717 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc397d90240, 0x8edb60, 0xc218543630, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc397f04060, 0x8ed920, 0xc2816d4dc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc396cbdc80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 2 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 2 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 2 minutes]:
net.(*pollDesc).Wait(0xc2080d8060, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080d8060, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080d8000, 0xc2080e3000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc2080d0430)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080cc010, 0xc2080e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc2080e4000)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc2080e4000, 0xc2080ec001, 0x5ff, 0x5ff, 0xc2080d6200, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc2080ce0e0, 0xc2080ec001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080e8000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e8000, 0x8f07a0, 0xc208066440, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc208066420, 0xc2080ce6e0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc20800bb40, 0x7fc636724c10, 0xc208066420, 0x0, 0x0, 0xc2080ce6e0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc20800bb40, 0x7fc636724c10, 0xc208066420, 0xc208086280, 0xc20801f580, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc20800bb40, 0x7fc636724c10, 0xc208066420)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc2080b3f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc2080b3f90, 0xc2080be480, 0x7fc636723398, 0xc2080ce020, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 67 [IO wait]:
net.(*pollDesc).Wait(0xc208010ed0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010ed0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010e70, 0xc208080000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc3a0e0d3d8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080cc2b8, 0xc208080000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc636724b08, 0xc2080cc2b8, 0xc20808acb8, 0xc208080000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080f3fa0, 0xc208080000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080673e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080673e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808ac60)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 19 [select, 2 minutes]:
net/http.(*persistConn).readLoop(0xc20808a0b0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 20 [select, 2 minutes]:
net/http.(*persistConn).writeLoop(0xc20808a0b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 14 [syscall, 2 minutes]:
syscall.Syscall(0x0, 0x0, 0xc20800f000, 0x1000, 0xc20803b1d0, 0xc2080ee3c0, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc20800f000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc20800f000, 0x1000, 0x1000, 0xc20803bc20, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208066060)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208066060, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 3013 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc211ef50e0, 0x3b9aca00, 0xc397ed50e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc397e7c4d0, 0x7fc63672c820, 0xc397d90240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fc63672c820, 0xc397d90240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 3012 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc211ef4f40, 0x3b9aca00, 0xc397ed50e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc397e7c4d0, 0x7fc63672caf0, 0xc396cbdc80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fc63672caf0, 0xc396cbdc80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 15 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc2080fe380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 16 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080fe390)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 33 [runnable]:
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 34 [select, 2 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 35 [select, 2 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2081540e0, 0x11, 0xc208063fc0, 0xc2080e49c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2081540e0, 0x11, 0xc208063fc0, 0xc2080e49c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2081540e0, 0x11, 0xc208063fc0, 0xc2080e49c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080db040)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 36 [select, 2 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080d1ed0, 0xf, 0xc208060fc0, 0xc2080e49c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080d1ed0, 0xf, 0xc208060fc0, 0xc2080e49c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080d1ed0, 0xf, 0xc208060fc0, 0xc2080e49c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2080db1d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 37 [select]:
net/http.(*Transport).getConn(0xe94600, 0xc3bd3d5e10, 0x0, 0xc212ef7260, 0x4, 0xc212ef7267, 0x11, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:525 +0x608
net/http.(*Transport).RoundTrip(0xe94600, 0xc3bd3d5e10, 0xc212ef72c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:228 +0x4d4
net/http.send(0xc3bd3d5e10, 0x7fc636722fe8, 0xe94600, 0x54, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3bd3d5e10, 0x54, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3bd3d5e10, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3bd3d5e10, 0xd, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3bd3d5e10, 0x7fc636724f78, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3bd4a2cb0, 0xc3ba5e2540, 0x3c, 0x7fc636724f78, 0xc3bd5680e0, 0x80b080, 0xc2127fbba0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc3b6a75df8, 0xc3ba5e2540, 0x3c, 0x80b080, 0xc2127fbba0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.(*Pool).refresh(0xc3b6a75dd0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:740 +0x1a2
github.com/couchbase/go-couchbase.(*Client).GetPool(0xc3bd534720, 0x9cae60, 0x7, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:771 +0x396
github.com/couchbase/goxdcr/utils.RemotePool(0xc2080ceb20, 0x11, 0xc2080d0740, 0xd, 0xc2080d0750, 0xc, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:98 +0x4c5
github.com/couchbase/goxdcr/utils.RemoteBucketUUID(0xc2080ceb20, 0x11, 0xc2080d0740, 0xd, 0xc2080d0750, 0xc, 0xc2080d1b30, 0x8, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:382 +0xa7
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080d81c0, 0xc3b6b37380, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:504 +0x101e
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080d81c0, 0xc3b6b37380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc2080d8bd0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 38 [chan send]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc2080e4d80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:110 +0x737
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 44 [IO wait, 2 minutes]:
net.(*pollDesc).Wait(0xc2080d8610, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080d8610, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080d85b0, 0xc208102000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc20802aa80)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080cc068, 0xc208102000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc636724b08, 0xc2080cc068, 0xc20808a108, 0xc208102000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080ce9c0, 0xc208102000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e43c0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080e43c0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080e43c0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080d5530)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080d5530, 0xc20816a000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080d7500, 0xc20816a000, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080d7500, 0xc20816a000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080d7540, 0xc20816a000, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc208116180, 0xc2080d6dc0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc208116180, 0x80f160, 0xc2080d7600, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 40 [select]:
net/http.(*persistConn).roundTrip(0xc20808a9a0, 0xc213aaddf0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc20802c090, 0xc31fc5c680, 0xc2129d01e0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc31fc5c680, 0x7fc636722fe8, 0xc20802c090, 0x1d, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d80, 0xc31fc5c680, 0x1d, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d80, 0xc31fc5c680, 0xb05680, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d80, 0xc31fc5c680, 0xc21263e360, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:177 +0x192
github.com/couchbase/cbauth/cbauthimpl.VerifyOnServer(0xc2080be4b0, 0xc21263e360, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:361 +0x283
github.com/couchbase/cbauth.doOnServer(0xc2080be4b0, 0xc21263e360, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/cbauth.go:126 +0x56
github.com/couchbase/cbauth.(*authImpl).AuthWebCreds(0xc2080380e8, 0xc3bd9512b0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/cbauth.go:162 +0x87
github.com/couchbase/cbauth.AuthWebCreds(0xc3bd9512b0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:132 +0x99
github.com/couchbase/goxdcr/replication_manager.authAdminCreds(0xc3bd9512b0, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:623 +0x5d
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc2080e4d80, 0xc3bd9512b0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:167 +0x14b0
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc2080e4d80, 0xc21241e990, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc21241e990, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc2080e4d98)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 41 [IO wait]:
net.(*pollDesc).Wait(0xc2080d9170, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080d9170, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080d9110, 0x0, 0x7fc636722df0, 0xc21241e6c0)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc2080cc110, 0xc20805ee38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc2080cc110, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc2080e5140, 0x7fc6367265a0, 0xc2080cc110, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 42 [select, 2 minutes]:
net/http.(*persistConn).readLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 43 [select, 2 minutes]:
net/http.(*persistConn).writeLoop(0xc20808a210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 45 [IO wait, 2 minutes]:
net.(*pollDesc).Wait(0xc2080d9250, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080d9250, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080d91f0, 0xc208135000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc20802b0e0)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080cc118, 0xc208135000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc636724b08, 0xc2080cc118, 0xc20808a268, 0xc208135000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208154540, 0xc208135000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e5320)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080e5320, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080e5320, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080d5620)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080d5620, 0xc20816a600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080d7640, 0xc20816a600, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080d7640, 0xc20816a600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080d7680, 0xc20816a600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc208116480, 0xc2080d6fc0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc208116480, 0x80f160, 0xc2080d7740, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 362 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xcc, 0xc214dcde00, 0xec, 0x100, 0xc22ab49800, 0xb7, 0x100, 0xc21ade42a0, 0xc21ade41e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1165 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0xcc, 0xc2a60bba00, 0xec, 0x100, 0xc2a692f800, 0xb7, 0x100, 0xc2a6cfa1e0, 0xc2a6cfa120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2355 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622efc0, 0xcb, 0xc36e11e700, 0xec, 0x100, 0xc36dd4b800, 0xc9, 0x100, 0xc36e162000, 0xc36e0a3f20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 68 [select]:
net/http.(*persistConn).writeLoop(0xc20808ac60)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 56 [runnable]:
net.(*pollDesc).Wait(0xc2080d8680, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080d8680, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080d8620, 0xc208134000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc3b197fed8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380c0, 0xc208134000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc636724b08, 0xc2080380c0, 0xc20808a9f8, 0xc208134000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081bf6e0, 0xc208134000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e4540)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e4540, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808a9a0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 57 [select]:
net/http.(*persistConn).writeLoop(0xc20808a9a0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2059 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a480, 0x2, 0xc2b115d300, 0xea, 0x100, 0xc325645800, 0xcf, 0x100, 0xc32591fd40, 0xc32591fc80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 334 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x5, 0xc214dcc600, 0xea, 0x100, 0xc21c228000, 0xbc, 0x100, 0xc21c97c180, 0xc21c97c0c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2993 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc2080e50e0, 0x7fc636726650, 0xc3bdb4e500, 0xc3bd951450)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080cc100, 0x7fc636726650, 0xc3bdb4e500, 0xc3bd951450)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fc636726650, 0xc3bdb4e500, 0xc3bd951450)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080e5140, 0x7fc636726650, 0xc3bdb4e500, 0xc3bd951450)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3bdb4e1e0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2992 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc2080e50e0, 0x7fc636726650, 0xc3bdb4e460, 0xc3bd951380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080cc100, 0x7fc636726650, 0xc3bdb4e460, 0xc3bd951380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fc636726650, 0xc3bdb4e460, 0xc3bd951380)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080e5140, 0x7fc636726650, 0xc3bdb4e460, 0xc3bd951380)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3bdb4e140)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 1471 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a8031b00, 0xca, 0xc2a60ba900, 0xec, 0x100, 0xc2c6bf6000, 0xbc, 0x100, 0xc2c6bc7da0, 0xc2c6bc7ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1731 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8fc0, 0x12d, 0xc308d35800, 0xec, 0x100, 0xc311452000, 0xcf, 0x100, 0xc311477e00, 0xc311477d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2997 [runnable]:
net.(*pollDesc).Wait(0xc211d16920, 0x77, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitWrite(0xc211d16920, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:93 +0x43
net.(*netFD).connect(0xc211d168c0, 0x0, 0x0, 0x7fc636723a20, 0xc2120a6960, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:114 +0x24b
net.(*netFD).dial(0xc211d168c0, 0x7fc6367239d0, 0x0, 0x7fc6367239d0, 0xc215f14690, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/sock_posix.go:139 +0x37a
net.socket(0x9d8aa0, 0x3, 0x2, 0x1, 0x0, 0xc215f14600, 0x7fc6367239d0, 0x0, 0x7fc6367239d0, 0xc215f14690, ...)
	/usr/local/go/src/net/sock_posix.go:91 +0x422
net.internetSocket(0x9d8aa0, 0x3, 0x7fc6367239d0, 0x0, 0x7fc6367239d0, 0xc215f14690, 0x0, 0x0, 0x0, 0x1, ...)
	/usr/local/go/src/net/ipsock_posix.go:137 +0x148
net.dialTCP(0x9d8aa0, 0x3, 0x0, 0xc215f14690, 0x0, 0x0, 0x0, 0xc212e94b40, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:156 +0x125
net.dialSingle(0x9d8aa0, 0x3, 0xc212ef7267, 0x11, 0x0, 0x0, 0x7fc636723940, 0xc215f14690, 0x0, 0x0, ...)
	/usr/local/go/src/net/dial.go:235 +0x3f5
net.func·016(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:163 +0x12c
net.dial(0x9d8aa0, 0x3, 0x7fc636723940, 0xc215f14690, 0xc2080b7b08, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/fd_unix.go:40 +0x6f
net.(*Dialer).Dial(0xc31ff3e940, 0x9d8aa0, 0x3, 0xc212ef7267, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:170 +0x4be
net.Dial(0x9d8aa0, 0x3, 0xc212ef7267, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/dial.go:143 +0x8a
net/http.(*Transport).dial(0xe94600, 0x9d8aa0, 0x3, 0xc212ef7267, 0x11, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:481 +0xcf
net/http.(*Transport).dialConn(0xe94600, 0x0, 0xc212ef7260, 0x4, 0xc212ef7267, 0x11, 0x439bde, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:564 +0x1678
net/http.func·019()
	/usr/local/go/src/net/http/transport.go:520 +0x42
created by net/http.(*Transport).getConn
	/usr/local/go/src/net/http/transport.go:522 +0x335

goroutine 2928 [runnable]:
time.Time.MarshalJSON(0xed09190db, 0x13b7f57d, 0xeb0f40, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/time/time.go:938 +0xf2
time.(*Time).MarshalJSON(0xc211ef4ea0, 0x0, 0x0, 0x0, 0x0, 0x0)
	<autogenerated>:34 +0xce
encoding/json.marshalerEncoder(0xc222b93290, 0x9b09e0, 0xc3aa5a88d0, 0xd9, 0x0)
	/usr/local/go/src/encoding/json/encode.go:416 +0x146
encoding/json.(*structEncoder).encode(0xc2080d5d10, 0xc222b93290, 0x92aa00, 0xc3aa5a88d0, 0xd9, 0x0)
	/usr/local/go/src/encoding/json/encode.go:584 +0x2ba
encoding/json.*structEncoder.(encoding/json.encode)·fm(0xc222b93290, 0x92aa00, 0xc3aa5a88d0, 0xd9, 0x92aa00)
	/usr/local/go/src/encoding/json/encode.go:598 +0x58
encoding/json.(*arrayEncoder).encode(0xc2080cc180, 0xc222b93290, 0x9006a0, 0xc211ef4e60, 0x57, 0xc2140d0800)
	/usr/local/go/src/encoding/json/encode.go:687 +0x11e
encoding/json.*arrayEncoder.(encoding/json.encode)·fm(0xc222b93290, 0x9006a0, 0xc211ef4e60, 0x57, 0x900600)
	/usr/local/go/src/encoding/json/encode.go:694 +0x58
encoding/json.(*sliceEncoder).encode(0xc2080cc188, 0xc222b93290, 0x9006a0, 0xc211ef4e60, 0x57, 0x0)
	/usr/local/go/src/encoding/json/encode.go:664 +0xb4
encoding/json.*sliceEncoder.(encoding/json.encode)·fm(0xc222b93290, 0x9006a0, 0xc211ef4e60, 0x57, 0xc211ef4e00)
	/usr/local/go/src/encoding/json/encode.go:673 +0x58
encoding/json.(*encodeState).reflectValue(0xc222b93290, 0x9006a0, 0xc211ef4e60, 0x57)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.(*encodeState).marshal(0xc222b93290, 0x9006a0, 0xc211ef4e60, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:270 +0xca
encoding/json.Marshal(0x9006a0, 0xc211ef4e60, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:135 +0x92
github.com/couchbase/goxdcr/pipeline.PipelineErrorArray.String(0xc3aa5a88d0, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/ReplicationStatus.go:61 +0x9f
github.com/couchbase/goxdcr/pipeline.(*ReplicationStatus).publishWithStatus(0xc2080dccf0, 0x9c2900, 0x7)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/ReplicationStatus.go:257 +0x1be
github.com/couchbase/goxdcr/pipeline.(*ReplicationStatus).Publish(0xc2080dccf0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/ReplicationStatus.go:236 +0x62
github.com/couchbase/goxdcr/pipeline.(*ReplicationStatus).SetPipeline(0xc2080dccf0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/ReplicationStatus.go:107 +0xdb
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).removePipelineFromReplicationStatus(0xeb0fa0, 0x7fc63672c900, 0xc397e7c4d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:321 +0xa8
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).stopPipeline(0xeb0fa0, 0xc2080dccf0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:349 +0x83f
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc3a956a180, 0xc3aa54dbc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:583 +0x274
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc3a956a180)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 2605 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f726480, 0x65, 0xc37edc5200, 0xec, 0x100, 0xc383e09000, 0xc6, 0x100, 0xc384261980, 0xc3842618c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 735 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0x12c, 0xc211161400, 0xec, 0x100, 0xc253ccc800, 0xac, 0x100, 0xc252515ce0, 0xc252554f60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 744 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0x12d, 0xc24ccfdc00, 0xec, 0x100, 0xc24dfa0800, 0xd0, 0x100, 0xc24f668d80, 0xc24f668cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2298 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622ed80, 0x64, 0xc218adb100, 0xec, 0x100, 0xc335ade800, 0xc9, 0x100, 0xc3362bb560, 0xc3362bb4a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 297 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12c, 0xc214dcdc00, 0xec, 0x100, 0xc217609000, 0xc4, 0x100, 0xc2182ad740, 0xc2182ad680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 739 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x0, 0xc24ccfc300, 0xea, 0x100, 0xc24c913800, 0x9f, 0x100, 0xc24ce943c0, 0xc24ce94300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1424 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a8031b00, 0xc8, 0xc2bd279f00, 0xec, 0x100, 0xc2b5a09000, 0xbb, 0x100, 0xc236734000, 0xc23676ff20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2279 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622efc0, 0x12c, 0xc222223700, 0xec, 0x100, 0xc3352b8000, 0xc4, 0x100, 0xc3352c5920, 0xc3352c5860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1065 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0x12d, 0xc27f8f3d00, 0xec, 0x100, 0xc260e18800, 0xcf, 0x100, 0xc262d001e0, 0xc262d00120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2329 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622efc0, 0x12e, 0xc35448b700, 0xec, 0x100, 0xc359b27800, 0x9e, 0x100, 0xc35a0e58c0, 0xc35a0e5800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2343 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622ed80, 0x67, 0xc35b7b0300, 0xec, 0x100, 0xc35a8ed000, 0xb3, 0x100, 0xc35b88e5a0, 0xc35b88e4e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1693 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8fc0, 0x12e, 0xc20802e900, 0xec, 0x100, 0xc2e3a43000, 0xb3, 0x100, 0xc2e4058960, 0xc2e40588a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2057 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a6c0, 0x132, 0xc2bd279c00, 0xec, 0x100, 0xc324e53000, 0xbf, 0x100, 0xc3251ef500, 0xc3251ef440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 116 [select]:
net/http.(*persistConn).writeLoop(0xc2080760b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1701 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8d80, 0x64, 0xc2eb746200, 0xec, 0x100, 0xc2eb54c000, 0xc9, 0x100, 0xc2eb709c20, 0xc2eb709b60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 115 [IO wait]:
net.(*pollDesc).Wait(0xc2083b6760, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2083b6760, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2083b6700, 0xc2083d7000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc3bcb479e8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080381c0, 0xc2083d7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc636724b08, 0xc2080381c0, 0xc208076108, 0xc2083d7000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20836a080, 0xc2083d7000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2083b8600)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2083b8600, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080760b0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2713 [runnable]:
github.com/couchbase/gomemcached/client.sendCommands(0xc39c7155c0, 0xc39c5b19e0, 0xc39c5b1a40)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 145 [syscall, 2 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 2317 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622ed80, 0x0, 0xc349daa800, 0xea, 0x100, 0xc349787000, 0x9d, 0x100, 0xc34a5186c0, 0xc34a518600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 149 [IO wait]:
net.(*pollDesc).Wait(0xc20c6e1800, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20c6e1800, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20c6e17a0, 0xc2083e7000, 0x1000, 0x1000, 0x0, 0x7fc636722df0, 0xc39c70d2b8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038140, 0xc2083e7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc636724b08, 0xc208038140, 0xc20808ad68, 0xc2083e7000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20c7b2cc0, 0xc2083e7000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2083da240)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2083da240, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20808ad10)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 150 [select]:
net/http.(*persistConn).writeLoop(0xc20808ad10)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 395 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x64, 0xc20816f700, 0xec, 0x100, 0xc234b76800, 0xc9, 0x100, 0xc235b715c0, 0xc235b71500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 402 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xca, 0xc20802f900, 0xec, 0x100, 0xc2339fb800, 0xbc, 0x100, 0xc233e8d380, 0xc233e8d2c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 353 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xcd, 0xc20802ea00, 0xec, 0x100, 0xc225d1b000, 0xba, 0x100, 0xc226af9800, 0xc226af9740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 816 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0x135, 0xc270edaa00, 0xec, 0x100, 0xc27108a000, 0xb9, 0x100, 0xc2710deb40, 0xc2710dea80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2955 [runnable]:
bufio.(*Writer).WriteString(0xc391853500, 0x9e1e30, 0xc, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:626
net/http.Header.WriteSubset(0xc218744ba0, 0x7fc636726798, 0xc3aa995320, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/header.go:156 +0x3e7
net/http.(*chunkWriter).writeHeader(0xc3b8430020, 0xc31d4ad000, 0x3b0, 0x800)
	/usr/local/go/src/net/http/server.go:883 +0xe16
net/http.(*chunkWriter).Write(0xc3b8430020, 0xc31d4ad000, 0x3b0, 0x800, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:256 +0x87
bufio.(*Writer).flush(0xc31ff3e1c0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:530 +0xe0
bufio.(*Writer).Flush(0xc31ff3e1c0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:519 +0x3a
net/http.(*response).finishRequest(0xc3b8430000)
	/usr/local/go/src/net/http/server.go:1018 +0x57
net/http.(*conn).serve(0xc3aa80a960)
	/usr/local/go/src/net/http/server.go:1208 +0xb91
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 364 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x68, 0xc218adab00, 0xec, 0x100, 0xc2284f3800, 0xc2, 0x100, 0xc222a88ba0, 0xc222a88ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 371 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x4, 0xc20802f400, 0xea, 0x100, 0xc22805b800, 0xa2, 0x100, 0xc22856ac60, 0xc22856aba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 332 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12d, 0xc211161000, 0xec, 0x100, 0xc217044000, 0xcf, 0x100, 0xc217e97e60, 0xc217e97da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 322 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x0, 0xc218adb400, 0xea, 0x100, 0xc213917000, 0xb8, 0x100, 0xc21a4f0600, 0xc21a4f0540, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1089 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0xc8, 0xc27073c800, 0xec, 0x100, 0xc282552000, 0xbb, 0x100, 0xc2826247e0, 0xc282624720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1359 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a80318c0, 0x0, 0xc2bd278d00, 0xea, 0x100, 0xc2bc1f0800, 0xb8, 0x100, 0xc2bd51ade0, 0xc2bd51ad20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2957 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc397d90480, 0x12c, 0xc3ab124a00, 0xec, 0x100, 0xc3aae7c800, 0x98, 0x100, 0xc3ab591d40, 0xc3ab591c80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 304 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x12e, 0xc20802f000, 0xec, 0x100, 0xc2166b1000, 0xb3, 0x100, 0xc216d18de0, 0xc216d18d20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3030 [runnable]:
sync.(*RWMutex).Lock(0xc397af7cb0)
	/usr/local/go/src/sync/rwmutex.go:87 +0xa4
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc397af7cb0, 0x1ead0108, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:309 +0x56
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc397d8a000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc397d8a000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc216e865a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1289 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a80318c0, 0x1, 0xc2b115d500, 0xe9, 0x100, 0xc2b1d43800, 0x2d, 0x40, 0xc2b1da3b60, 0xc2b1da3aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 790 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0xc9, 0xc210e0bc00, 0xec, 0x100, 0xc25dcd7000, 0xbd, 0x100, 0xc25e72f9e0, 0xc25e72f920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3031 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc397d90480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc397d90480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc216e86660)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2795 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc3a0de4a50, 0xc3a0a2d3e0, 0xc3a0a2d440)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 1752 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8d80, 0x65, 0xc312746a00, 0xec, 0x100, 0xc312e21800, 0xb4, 0x100, 0xc3130c3c80, 0xc3130c3bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 788 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x6, 0xc210e0a600, 0xea, 0x100, 0xc25c1b7000, 0xbe, 0x100, 0xc25ce3c720, 0xc25ce3c660, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 995 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0x12c, 0xc27ba80d00, 0xec, 0x100, 0xc27b4fd000, 0x44, 0x80, 0xc27bfa8540, 0xc27bfa8480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3014 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc211ef52e0, 0x3b9aca00, 0xc397ed50e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc397e7c4d0, 0x7fc63672caf0, 0xc397d8a000, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fc63672caf0, 0xc397d8a000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 762 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0xd0, 0xc21fd27900, 0xec, 0x100, 0xc245646000, 0xb2, 0x100, 0xc244efd920, 0xc244efd860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 289 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x65, 0xc210e0bb00, 0xec, 0x100, 0xc212e4b000, 0xb4, 0x100, 0xc213557b60, 0xc213557aa0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2614 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f7266c0, 0x12c, 0xc383dca100, 0xec, 0x100, 0xc383b31800, 0xc4, 0x100, 0xc383ce3f20, 0xc383ce3e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 709 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x64, 0xc222223300, 0xec, 0x100, 0xc23a6c5800, 0xc9, 0x100, 0xc210e51c80, 0xc210e51bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 265 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0xcb, 0xc20802fb00, 0xec, 0x100, 0xc212a57800, 0xb5, 0x100, 0xc2135e74a0, 0xc2135e73e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1427 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a8031b00, 0x12c, 0xc2bda01d00, 0xec, 0x100, 0xc2c09f1800, 0xcb, 0x100, 0xc2c1232c60, 0xc2c1232ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 309 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805c480, 0x2, 0xc218adbb00, 0xea, 0x100, 0xc2197e6800, 0xcf, 0x100, 0xc219e31200, 0xc219e31140, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2029 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a6c0, 0x12e, 0xc31d132900, 0xec, 0x100, 0xc30c3d2800, 0x9d, 0x100, 0xc28a4ef920, 0xc28a4ef860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2758 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc397d8a000)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2031 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a480, 0x0, 0xc31d133c00, 0xea, 0x100, 0xc30b41c800, 0x9f, 0x100, 0xc287dc3ec0, 0xc287dc3e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 811 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0x134, 0xc26906ee00, 0xec, 0x100, 0xc268868800, 0xc6, 0x100, 0xc26935f4a0, 0xc26935f3e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 796 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x66, 0xc264967e00, 0xec, 0x100, 0xc266e55000, 0xc8, 0x100, 0xc267b128a0, 0xc267b127e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1998 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a6c0, 0x12c, 0xc27f8f3500, 0xec, 0x100, 0xc2a5e88800, 0xc4, 0x100, 0xc29eddb800, 0xc29eddb740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1898 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a6c0, 0x12e, 0xc31da32800, 0xeb, 0x100, 0xc31da5b800, 0x20, 0x20, 0xc31daac3c0, 0xc31daac300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1894 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a480, 0x0, 0xc31d133900, 0xea, 0x100, 0xc31d0e1800, 0x3f, 0x40, 0xc31d2328a0, 0xc31d2327e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 713 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2240, 0xc8, 0xc21fd27100, 0xec, 0x100, 0xc239784800, 0xbb, 0x100, 0xc21d9bc2a0, 0xc21d9bc1e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2361 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622ed80, 0x2, 0xc36e60d400, 0xea, 0x100, 0xc36e89c000, 0xcf, 0x100, 0xc36e90f4a0, 0xc36e90f3e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 826 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x1, 0xc27073d000, 0xea, 0x100, 0xc2705ed800, 0xc7, 0x100, 0xc270a0c5a0, 0xc270a0c4e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1058 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x65, 0xc27f8f3400, 0xec, 0x100, 0xc22ac8b000, 0x99, 0x100, 0xc2132000c0, 0xc213200000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 577 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x0, 0xc20816e300, 0xea, 0x100, 0xc23904c600, 0x3f, 0x40, 0xc234d87860, 0xc234d877a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 733 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x68, 0xc211161100, 0xec, 0x100, 0xc245421800, 0xc2, 0x100, 0xc221e7b680, 0xc221dc6de0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2714 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc396cbdc80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2512 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f7266c0, 0xc8, 0xc378c0a200, 0xeb, 0x100, 0xc378be2600, 0x23, 0x40, 0xc378c3e780, 0xc378c3e6c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1718 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8fc0, 0xc8, 0xc2f00e5f00, 0xec, 0x100, 0xc2f73bc800, 0xbb, 0x100, 0xc2f7e75ec0, 0xc2f7e75e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 760 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc22f9c2000, 0x65, 0xc21fd26400, 0xec, 0x100, 0xc24d539800, 0xc6, 0x100, 0xc2376dd020, 0xc2376dcf60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1073 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0x12c, 0xc27ba81a00, 0xec, 0x100, 0xc25c494000, 0x98, 0x100, 0xc25c52ed80, 0xc25c52ecc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3011 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc2080e50e0, 0x7fc636726650, 0xc3bdb4e640, 0xc3bd9515f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080cc100, 0x7fc636726650, 0xc3bdb4e640, 0xc3bd9515f0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fc636726650, 0xc3bdb4e640, 0xc3bd9515f0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080e5140, 0x7fc636726650, 0xc3bdb4e640, 0xc3bd9515f0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3bdb4e320)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 1420 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a80318c0, 0x64, 0xc2bd278000, 0xec, 0x100, 0xc285665000, 0xc9, 0x100, 0xc292f056e0, 0xc292f05620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 979 [chan send, 2 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x65, 0xc27ab15a00, 0xec, 0x100, 0xc27b8de400, 0x6a, 0x80, 0xc27b9bcf60, 0xc27b9bcea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1119 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0x130, 0xc21fd26800, 0xec, 0x100, 0xc28d3f2000, 0xbc, 0x100, 0xc28d4f24e0, 0xc28d4f2420, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1093 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x0, 0xc271f3f800, 0xea, 0x100, 0xc259174000, 0xb8, 0x100, 0xc264a94960, 0xc264a948a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1079 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x1, 0xc27e20d800, 0xea, 0x100, 0xc27fd9c000, 0xc7, 0x100, 0xc25856e960, 0xc25856e8a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1115 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x64, 0xc24ccfd800, 0xec, 0x100, 0xc28bf97800, 0xc9, 0x100, 0xc28c1e0d80, 0xc28c1e0cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2039 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a6c0, 0xc8, 0xc2eb747400, 0xec, 0x100, 0xc2f7b11800, 0xbb, 0x100, 0xc289e982a0, 0xc289e981e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1178 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x68, 0xc2a65a2200, 0xec, 0x100, 0xc2a61b6800, 0xc2, 0x100, 0xc2a654d800, 0xc2a654d740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1985 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a480, 0x64, 0xc24ccfcf00, 0xec, 0x100, 0xc299a0a000, 0xc9, 0x100, 0xc29f9fc660, 0xc29f9fc5a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1137 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0xc9, 0xc20802eb00, 0xec, 0x100, 0xc2a4ab3800, 0xbd, 0x100, 0xc2a4e9b020, 0xc2a4e9af60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1135 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x5, 0xc210e0b000, 0xea, 0x100, 0xc2a3fcb000, 0xbc, 0x100, 0xc2a474ac60, 0xc2a474aba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1141 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc271a9bd40, 0x3, 0xc222222300, 0xea, 0x100, 0xc297268800, 0xb5, 0x100, 0xc298522c00, 0xc298522b40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1145 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2721f6000, 0xca, 0xc218ada300, 0xec, 0x100, 0xc29981b800, 0xbc, 0x100, 0xc299f18060, 0xc299d1df80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1601 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8fc0, 0x12c, 0xc2695bd700, 0xec, 0x100, 0xc2d1e3f000, 0x85, 0x100, 0xc2d1f37620, 0xc2d1f37560, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3015 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc211ef5440, 0x3b9aca00, 0xc397ed50e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc397e7c4d0, 0x7fc63672c820, 0xc397d90480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7fc63672c820, 0xc397d90480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 1335 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a8031b00, 0x12c, 0xc2b2398f00, 0xec, 0x100, 0xc2b271c000, 0xb7, 0x100, 0xc2b27b6d80, 0xc2b27b6cc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2043 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc31390a480, 0x65, 0xc2bda00c00, 0xec, 0x100, 0xc2e23e5800, 0xc6, 0x100, 0xc2de50b2c0, 0xc2de50b200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2656 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f7266c0, 0xc8, 0xc395d0cb00, 0xec, 0x100, 0xc3962b7000, 0xa8, 0x100, 0xc3966d1e60, 0xc3966d1da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1709 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8d80, 0x2, 0xc300e83300, 0xea, 0x100, 0xc300a77800, 0xcf, 0x100, 0xc301131c80, 0xc301131bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1671 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8d80, 0x66, 0xc225a1fc00, 0xec, 0x100, 0xc2d6944800, 0xc8, 0x100, 0xc2d70fcd80, 0xc2d70fccc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2757 [semacquire]:
sync.(*WaitGroup).Wait(0xc397d903e0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc397d90240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc397d90240)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1596 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2c7fb8d80, 0x65, 0xc2695bc700, 0xeb, 0x100, 0xc2d1219200, 0x17, 0x20, 0xc2d11ecae0, 0xc2d11eca20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1448 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a80318c0, 0x1, 0xc2a7f8ad00, 0xea, 0x100, 0xc2b175d000, 0xae, 0x100, 0xc2712fd260, 0xc2712fd1a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1478 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2a80318c0, 0x7, 0xc27ab15100, 0xea, 0x100, 0xc2c6bf9000, 0xbe, 0x100, 0xc2c786aa20, 0xc2c786a960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3010 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc2080e50e0, 0x7fc636726650, 0xc3bdb4e5a0, 0xc3bd951520)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080cc100, 0x7fc636726650, 0xc3bdb4e5a0, 0xc3bd951520)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fc636726650, 0xc3bdb4e5a0, 0xc3bd951520)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080e5140, 0x7fc636726650, 0xc3bdb4e5a0, 0xc3bd951520)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3bdb4e280)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2530 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f726480, 0x64, 0xc378c0a500, 0xec, 0x100, 0xc378be3200, 0x3a, 0x40, 0xc378e24780, 0xc378e246c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2756 [runnable]:
encoding/json.(*decodeState).scanWhile(0xc27eb6d2c0, 0x9, 0xc218744180)
	/usr/local/go/src/encoding/json/decode.go:229
encoding/json.(*decodeState).objectInterface(0xc27eb6d2c0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:873 +0x5f
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc27eb6d2c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc27eb6d2c0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).objectInterface(0xc27eb6d2c0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc27eb6d2c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc27eb6d2c0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).objectInterface(0xc27eb6d2c0, 0x9)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:834 +0x84
encoding/json.(*decodeState).arrayInterface(0xc27eb6d2c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:854 +0x117
encoding/json.(*decodeState).valueInterface(0xc27eb6d2c0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:832 +0xae
encoding/json.(*decodeState).objectInterface(0xc27eb6d2c0, 0xc213aacd30)
	/usr/local/go/src/encoding/json/decode.go:900 +0x218
encoding/json.(*decodeState).object(0xc27eb6d2c0, 0x833e80, 0xc213aacd30, 0xd4)
	/usr/local/go/src/encoding/json/decode.go:497 +0x3c8
encoding/json.(*decodeState).value(0xc27eb6d2c0, 0x812be0, 0xc213aacd30, 0x16)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).unmarshal(0xc27eb6d2c0, 0x812be0, 0xc213aacd30, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:149 +0x1e7
encoding/json.Unmarshal(0xc3a1bcaf43, 0x469, 0x469, 0x812be0, 0xc213aacd30, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:78 +0x145
github.com/couchbase/goxdcr/simple_utils.IsJSON(0xc3a1bcaf43, 0x469, 0x469, 0xc2d094b3a8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:168 +0x6c
github.com/couchbase/goxdcr/parts.getDocMap(0xc37aa2ec60, 0xc31f943b00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:987 +0x4fe
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc397d90240, 0xa90000, 0xc2d094bb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc397d90240, 0xc3a1f00000, 0xc2d094bb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc397d90240, 0xc39c478b60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc397d90240, 0xc39c478b60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc397d90240, 0xc397df1110, 0xc397d903e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 2760 [semacquire]:
sync.(*RWMutex).RLock(0xc397af7cb0)
	/usr/local/go/src/sync/rwmutex.go:36 +0x5f
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc397af7cb0, 0xc3a0b75680)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:424 +0x4a7
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 2311 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622efc0, 0x12d, 0xc210e0be00, 0xec, 0x100, 0xc345e9a000, 0xd0, 0x100, 0xc345ec3bc0, 0xc345ec3b00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2191 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622ed80, 0x0, 0xc27ab14300, 0xea, 0x100, 0xc32f1e9800, 0x3f, 0x40, 0xc32f22be60, 0xc32f22bda0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2660 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc36f726480, 0x64, 0xc395902100, 0xec, 0x100, 0xc395681800, 0xab, 0x100, 0xc396872ba0, 0xc396872ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3028 [chan send]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc397af70e0, 0x1eae0019, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc396cbdc80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc396cbdc80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc216e86420)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2771 [semacquire]:
sync.(*WaitGroup).Wait(0xc397d90620)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc397d90480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc397d90480)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 2227 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc32622efc0, 0x12e, 0xc271f3ec00, 0xeb, 0x100, 0xc32f6af800, 0x1f, 0x20, 0xc32fa16420, 0xc32fa16360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2991 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc2080e50e0, 0x7fc636726650, 0xc3bdb4e3c0, 0xc3bd9512b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080cc100, 0x7fc636726650, 0xc3bdb4e3c0, 0xc3bd9512b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fc636726650, 0xc3bdb4e3c0, 0xc3bd9512b0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080e5140, 0x7fc636726650, 0xc3bdb4e3c0, 0xc3bd9512b0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3bdb4e0a0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 2954 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc397d90240, 0x64, 0xc3a5387d00, 0xec, 0x100, 0xc3a9b82800, 0xab, 0x100, 0xc3aa962d20, 0xc3aa962c60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2716 [select]:
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc397af70e0, 0xc39c5b1f80)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:547 +0xa0e
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 2976 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc397d90240, 0x0, 0xc3b95f7f00, 0xea, 0x100, 0xc3b8f93800, 0xb8, 0x100, 0xc3b9a8eba0, 0xc3b9a8eae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2770 [runnable]:
bytes.func·001()
	/usr/local/go/src/bytes/buffer.go:186
bytes.makeSlice(0xd00, 0xc245afba00, 0xd00, 0xd00)
	/usr/local/go/src/bytes/buffer.go:191 +0x98
bytes.(*Buffer).grow(0xc224d829a0, 0x400, 0x400)
	/usr/local/go/src/bytes/buffer.go:99 +0x243
bytes.(*Buffer).Write(0xc224d829a0, 0xc21aa76038, 0x400, 0x400, 0x0, 0x0, 0x0)
	/usr/local/go/src/bytes/buffer.go:127 +0x52
encoding/base64.(*encoder).Write(0xc21aa76000, 0xc3a29d2343, 0x5f8, 0x5f8, 0x300, 0x0, 0x0)
	/usr/local/go/src/encoding/base64/base64.go:163 +0x449
encoding/json.encodeByteSlice(0xc224d829a0, 0x8271c0, 0xc212918520, 0x57, 0xc212918500)
	/usr/local/go/src/encoding/json/encode.go:648 +0x3ac
encoding/json.(*encodeState).reflectValue(0xc224d829a0, 0x8271c0, 0xc212918520, 0x57)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.interfaceEncoder(0xc224d829a0, 0x833e80, 0xc22d623600, 0x54, 0xc22d623500)
	/usr/local/go/src/encoding/json/encode.go:557 +0xd9
encoding/json.(*mapEncoder).encode(0xc208038060, 0xc224d829a0, 0x8374c0, 0xc21bb0a8a0, 0x15, 0x0)
	/usr/local/go/src/encoding/json/encode.go:619 +0x33d
encoding/json.*mapEncoder.(encoding/json.encode)·fm(0xc224d829a0, 0x8374c0, 0xc21bb0a8a0, 0x15, 0xc21bb0a800)
	/usr/local/go/src/encoding/json/encode.go:629 +0x58
encoding/json.(*encodeState).reflectValue(0xc224d829a0, 0x8374c0, 0xc21bb0a8a0, 0x15)
	/usr/local/go/src/encoding/json/encode.go:299 +0x72
encoding/json.(*encodeState).marshal(0xc224d829a0, 0x8374c0, 0xc21bb0a8a0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:270 +0xca
encoding/json.Marshal(0x8374c0, 0xc21bb0a8a0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/encode.go:135 +0x92
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc397d90480, 0x4000cc, 0xc3ab1a5b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:785 +0x34b
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc397d90480, 0xc3a2b600cc, 0xc3ab1a5b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc397d90480, 0xc3a08c0230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc397d90480, 0xc3a08c0230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc397d90480, 0xc397df1260, 0xc397d90620, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 3029 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc397d90240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc397d90240, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc216e864e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 2990 [chan send]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc2080e50e0, 0x7fc636726650, 0xc3bdb4e780, 0xc3bd9511e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:135 +0x190
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080cc100, 0x7fc636726650, 0xc3bdb4e780, 0xc3bd9511e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7fc636726650, 0xc3bdb4e780, 0xc3bd9511e0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080e5140, 0x7fc636726650, 0xc3bdb4e780, 0xc3bd9511e0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3bdb4e000)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e
[goport] 2017/04/25 13:48:34 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:48:35.051-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:48:35.051-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:48:35.052-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:48:35.079-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:48:35.084-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:48:35.084-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:48:35.084-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:48:35.085-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:48:35.085-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:48:35.085-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:48:35.085-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:48:35.085-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:48:35.085-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationSpecChangeListener 2017-04-25T13:48:35.085-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:48:35.085-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:48:35.085-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:48:35.085-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:48:35.085-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:48:35.085-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:48:35.085-05:00 [INFO] ReplicationManager is running
ReplicationManager 2017-04-25T13:48:35.085-05:00 [INFO] pollEOF: About to start stdin polling
HttpServer 2017-04-25T13:48:35.087-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:48:35.087-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:48:35.087-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
ReplicationSpecChangeListener 2017-04-25T13:48:35.091-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:48:35.091-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
RemoteClusterChangeListener 2017-04-25T13:48:35.093-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:48:35.093-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
PipelineManager 2017-04-25T13:48:35.396-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:48:50.570-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:48:50.924-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:48:51.026-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:48:51.027-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:48:51.027-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:48:51.069-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:48:51.069-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:48:51.069-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:48:51.069-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:48:51.070-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:48:51.070-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:48:51.070-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:48:51.072-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:48:51.077-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
GenericSupervisor 2017-04-25T13:48:51.079-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:48:51.079-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
GenericSupervisor 2017-04-25T13:48:51.079-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:48:51.079-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:48:51.079-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:48:51.079-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:48:51.079-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:48:51.079-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
CheckpointManager 2017-04-25T13:48:51.079-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
CapiUtils 2017-04-25T13:48:51.105-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:48:51.105-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:48:51.105-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:48:51.111-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
StatisticsManager 2017-04-25T13:48:51.111-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:48:51.111-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
XDCRFactory 2017-04-25T13:48:51.111-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:48:51.112-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011 is constructed, start it
GenericPipeline 2017-04-25T13:48:51.113-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[229 277 236 257 268 195 186 315 221 266 281 325 329 192 237 183 258 285 296 303 311 255 328 204 213 279 340 216 332 172 207 214 336 174 275 288 171 227 198 234 259 212 238 242 249 289 318 320 232 276 177 278 294 253 262 323 312 317 339 338 206 233 248 239 322 225 270 284 319 226 273 300 181 211 220 286 324 269 219 264 287 295 187 200 254 293 197 189 209 230 304 313 314 185 201 175 196 180 244 335 193 205 240 241 179 309 316 176 210 251 223 298 330 321 182 203 231 250 178 208 224 272 333 215 199 331 173 247 265 290 292 301 184 327 217 302 305 188 299 308 191 235 326 228 246 260 261 337 271 218 243 252 190 202 274 297 280 194 267 256 307 222 245 282 283 306 263 291 310 334] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[140 106 158 72 92 150 160 55 133 39 149 24 125 12 75 109 6 79 126 27 58 104 129 152 36 64 145 53 103 7 57 170 42 52 78 81 117 35 123 26 50 96 144 5 128 139 151 154 159 112 134 137 161 32 148 16 47 105 30 167 156 80 68 102 116 138 25 169 165 63 54 77 127 114 31 95 162 163 10 18 49 66 2 118 142 155 71 90 119 69 89 84 38 83 97 14 17 40 60 157 11 51 93 46 61 101 135 28 107 120 168 73 33 21 88 115 1 45 94 37 23 44 56 99 8 85 19 74 48 59 20 124 41 43 91 113 143 4 86 100 22 76 111 130 147 166 65 110 15 98 131 0 34 136 29 87 108 153 13 146 141 70 121 164 3 67 62 82 132 9 122]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[15 110 0 98 131 29 34 136 153 13 87 108 141 146 164 3 70 121 62 67 9 82 132 122 140 72 106 158 160 55 92 150 39 133 149 12 24 125 75 6 109 79 27 126 129 152 36 58 104 53 64 145 7 103 57 42 170 81 117 35 52 78 26 123 144 5 50 96 151 154 159 112 128 139 161 32 134 137 16 148 30 47 105 167 80 156 116 138 25 68 102 169 165 54 63 77 114 127 162 163 10 31 95 66 2 18 49 155 71 118 142 69 90 119 84 89 97 14 38 83 60 157 11 17 40 46 51 93 135 28 61 101 168 73 107 120 33 115 1 21 88 37 45 94 56 99 8 23 44 85 19 48 74 20 59 124 91 113 143 4 41 43 22 86 100 130 147 166 65 76 111] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[297 280 194 267 307 256 222 245 282 283 306 291 263 310 334 229 277 268 195 236 257 186 315 221 266 281 325 329 237 192 285 296 303 311 183 258 255 328 279 340 204 213 332 216 214 336 172 207 275 288 174 171 227 198 234 259 212 249 289 318 320 238 242 276 232 177 278 294 262 323 253 312 317 339 338 206 233 248 322 239 225 270 284 319 300 226 273 211 220 286 324 181 269 287 295 219 264 200 254 293 187 197 230 304 313 314 185 189 209 201 175 196 180 244 335 193 205 240 241 179 309 316 210 251 176 223 298 330 321 182 203 231 250 178 208 224 272 333 215 199 331 173 247 265 290 292 301 184 327 305 217 302 308 188 299 235 326 191 260 261 337 228 246 271 218 243 252 190 202 274]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 stats_interval:1000 log_level:Info replication_type:capi filter_expression: active:true doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:48:51.113-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011...
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:48:51.122-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:48:51.122-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:48:51.122-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:48:51.122-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:48:51.122-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:48:51.122-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:48:51.122-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:48:51.122-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:48:51.122-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:48:51.122-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:48:51.124-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:48:51.124-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:48:51.124-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:48:51.124-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:48:51.125-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiUtils 2017-04-25T13:48:51.155-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:48:51.156-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:48:51.156-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:48:51.189-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:48:51.191-05:00 [INFO] validateSourceTopology...
TCPConnPoolMgr 2017-04-25T13:48:51.320-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:48:51.321-05:00 [INFO] connectionSize=5
TCPConnPoolMgr 2017-04-25T13:48:51.323-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:48:51.323-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:48:51.323-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:48:51.323-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:48:51.351-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:48:51.351-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:48:51.391-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:48:51.412-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:48:51.464-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:48:51.464-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:48:51.464-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:48:51.465-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:48:51.465-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:48:51.465-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:48:51.550-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:48:51.550-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:48:51.551-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:48:51.572-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:48:51.572-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:48:51.660-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:48:51.660-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:48:51.660-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:48:51.660-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:48:51.712-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:48:51.712-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:48:51.712-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:48:51.733-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:48:51.733-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:48:51.733-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:48:51.733-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011 is started----------
PipelineManager 2017-04-25T13:48:51.733-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:48:51.733-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:48:51.733-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:48:51 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:48:51 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 309 successful
CapiNozzle 2017-04-25T13:48:52.133-05:00 [INFO] Send batch count=205
StatisticsManager 2017-04-25T13:48:52.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:48:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 243 successful
CapiNozzle 2017-04-25T13:48:52.396-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:48:52.406-05:00 [INFO] Send batch count=43
CapiNozzle 2017-04-25T13:48:52.446-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:48:52 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 149 successful
CapiNozzle 2017-04-25T13:48:52.457-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
2017/04/25 13:48:52 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 252 successful
CapiNozzle 2017-04-25T13:48:52.517-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:52.530-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:48:52 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 281 successful
CapiNozzle 2017-04-25T13:48:52.752-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:48:52.753-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:52.767-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
2017/04/25 13:48:52 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:48:52.784-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:48:52 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:48:52.976-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
CheckpointManager 2017-04-25T13:48:52.977-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:48:52 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:48:52 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:48:53 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:48:53 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:48:53 UPR_STREAMREQ for vb 199 successful
CapiNozzle 2017-04-25T13:48:53.079-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:53.091-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
DcpNozzle 2017-04-25T13:48:53.138-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:48:53.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:48:53.180-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:48:53.266-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:48:53.277-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:48:53.281-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:53.281-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:48:53.281-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:48:53.281-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
ReplicationManager 2017-04-25T13:48:53.281-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
PipelineManager 2017-04-25T13:48:53.289-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:48:53.290-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:48:53.293-05:00 [INFO] err_list=[{"time":"2017-04-25T13:48:53.293752249-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]
PipelineManager 2017-04-25T13:48:53.294-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:48:53.373-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:53.373-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:53.385-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:48:53.571-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:48:53.571-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:48:53.571-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:48:53.571-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:53.571-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:53.638-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:53.638-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:53.649-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:48:53.814-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:53.859-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:48:53.995-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:48:54.014-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
StatisticsManager 2017-04-25T13:48:54.124-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:48:54.124-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011 {"Errors": "[{\"time\":\"2017-04-25T13:48:53.293752249-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 16, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 99, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 16677, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 99, "set_filtered": 0, "set_received_from_dcp": 16677, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:48:54.252-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:54.263-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:48:54.430-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:48:54.448-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:48:54.635-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:48:54.635-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:48:54.635-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:48:54.635-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:54.635-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:54.725-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:54.725-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:54.736-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:48:54.815-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:48:54.815-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:48:54.815-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:48:54.815-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:54.815-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:54.917-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:54.917-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:54.946-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:48:55.054-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:48:55.070-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:48:55.323-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:48:55.335-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:48:55.620-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:48:55.638-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:48:55.910-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:48:55.928-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:48:56.102-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:48:56.102-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:48:56.102-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:48:56.102-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:56.102-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:56.213-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:56.213-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:56.253-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:48:56.279-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:48:56.279-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:48:56.279-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:48:56.279-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:56.279-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:56.372-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:56.372-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:56.407-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:48:56.664-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:48:56.687-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:48:56.880-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:48:56.901-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:48:57.309-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:48:57.331-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:48:57.649-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:48:57.671-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:48:57.956-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:48:57.956-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:48:57.956-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:48:57.956-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:57.956-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:58.027-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:58.027-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:58.042-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:48:58.089-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:48:58.089-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:48:58.089-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:48:58.089-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:58.089-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:58.147-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:58.147-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:58.167-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:48:58.605-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:48:58.625-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:48:58.955-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:48:58.978-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:48:59.304-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:48:59.326-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:48:59.495-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:48:59.526-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:48:59.803-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:48:59.803-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:48:59.803-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:48:59.803-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:48:59.803-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:48:59.873-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:48:59.873-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:48:59.888-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:49:00.317-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:00.317-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:49:00.317-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:49:00.317-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:00.317-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:00.423-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:00.423-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:00.439-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:49:00.588-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:49:00.605-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:49:00.877-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:00.909-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:49:01.122-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
GenericSupervisor 2017-04-25T13:49:01.125-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:49:01.125-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:49:01.125-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:49:01.162-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
ToplogyChangeDetector 2017-04-25T13:49:01.423-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:49:01.454-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:01.480-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:49:01.741-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:49:01.741-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:49:01.741-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:49:01.741-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:01.741-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:01.820-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:01.820-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:01.841-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:49:02.015-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:02.021-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:49:02.022-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:49:02.022-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:02.022-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:02.109-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:02.109-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:02.131-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:49:02.228-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:02.248-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:49:02.501-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:49:02.521-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:49:02.668-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:02.679-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:49:02.854-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:49:02.865-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:49:03.167-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:03.167-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:49:03.167-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:49:03.167-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:03.167-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:49:03.302-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries 
PipelineManager 2017-04-25T13:49:03.302-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:49:03.302-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:03.302-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
PipelineRuntimeCtx 2017-04-25T13:49:03.302-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:49:03.302-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:03.302-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:49:03.302-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:03.302516529 -0500 CDT
GenericSupervisor 2017-04-25T13:49:03.302-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:03.302-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:49:03.302-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:03.302-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:49:03.302-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:03.302-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:49:03.302-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:49:03.302-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:49:03.302-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:49:03.303-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-70109011 has been stopped
GenericPipeline 2017-04-25T13:49:03.303-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-70109011 is stopped
PipelineManager 2017-04-25T13:49:03.303-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:49:03.303-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:48:53.293752249-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:49:03.303-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:03.303-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
GenericPipeline 2017-04-25T13:49:03.304-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
GenericPipeline 2017-04-25T13:49:03.304-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
GenericPipeline 2017-04-25T13:49:03.304-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-70109011
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:03.304-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:03.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:49:03.305-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:03.305-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:49:03.305-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:03.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:49:03.305-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:03.305-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:49:03.305-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:03.305-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:03.305-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:49:03.305-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:03.30538891 -0500 CDT
DcpNozzle 2017-04-25T13:49:03.306-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:49:03.306-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:03.305690368 -0500 CDT
CapiNozzle 2017-04-25T13:49:03.306-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:49:03.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:03.306-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:03.306-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:03.306-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:03.307-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:49:03.307-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:03.307-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:49:03.310-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:03.310-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:03.310-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:03.310123171 -0500 CDT
DcpNozzle 2017-04-25T13:49:03.310-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:03.310141064 -0500 CDT
DcpNozzle 2017-04-25T13:49:03.310-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:49:03.310-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:49:03.310-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:03.310-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:49:03.310-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:03.310-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:49:03 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:49:03 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:49:03 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:49:03.358-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:49:03.358-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:49:03.358-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:03.358-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:49:03.472-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:03.472-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:49:03.473-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:49:03.516-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:03.517-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:49:03.517-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:03.517-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:49:03.517-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:03.517-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:49:03.517-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:49:03.518-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:49:03.518-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
GenericSupervisor 2017-04-25T13:49:03.520-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:49:03.520-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
GenericSupervisor 2017-04-25T13:49:03.520-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:03.520-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:03.520-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:03.520-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:03.520-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:49:03.520-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
CheckpointManager 2017-04-25T13:49:03.520-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
CapiUtils 2017-04-25T13:49:03.548-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:03.548-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:03.548-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:49:03.549-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
StatisticsManager 2017-04-25T13:49:03.550-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:49:03.550-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
XDCRFactory 2017-04-25T13:49:03.550-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:49:03.551-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701 is constructed, start it
GenericPipeline 2017-04-25T13:49:03.552-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[2 22 54 156 4 13 115 136 5 110 161 155 162 1 113 37 120 26 141 47 67 165 30 41 6 131 145 147 159 86 0 56 123 154 15 16 128 132 148 168 63 80 144 20 70 105 133 36 52 66 17 64 103 149 160 46 49 51 71 142 157 27 83 90 96 150 10 59 119 137 170 12 34 99 32 38 68 98 130 24 78 87 108 114 19 44 57 69 74 81 21 72 73 75 104 23 25 166 135 7 33 39 122 55 95 18 43 62 45 117 31 35 111 125 134 8 169 28 53 143 84 139 85 93 94 107 164 101 9 11 92 158 100 116 124 138 152 14 77 129 40 60 146 65 91 42 79 118 61 163 29 82 106 126 127 153 50 76 88 109 89 102 121 140 48 97 112 3 167 58 151] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[321 237 251 303 206 243 310 175 254 285 227 228 270 299 238 258 322 190 250 281 318 306 173 283 313 183 255 327 217 186 189 212 236 252 309 248 316 330 259 194 288 311 323 177 253 328 276 231 279 208 209 304 244 181 229 220 198 210 293 338 287 334 340 178 264 271 185 286 302 202 214 265 274 284 291 296 180 242 200 196 201 260 292 172 213 312 336 235 256 301 315 324 257 314 222 226 305 199 223 240 298 337 192 224 326 268 325 261 267 333 332 230 273 295 241 278 317 184 187 232 239 247 193 216 300 207 307 329 182 203 218 272 282 176 179 219 221 204 233 262 188 195 249 320 339 205 225 269 275 215 280 245 290 197 289 308 335 174 246 263 266 294 319 331 171 191 234 297 211 277]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[72 73 75 104 23 25 166 135 7 33 39 122 55 95 18 43 62 45 117 31 35 111 125 134 8 169 28 53 143 84 139 85 93 94 107 164 101 9 11 92 158 100 116 124 138 152 14 77 129 40 60 146 65 91 42 79 118 61 163 29 82 106 126 127 153 50 76 88 109 89 102 121 140 48 97 112 3 167 58 151 2 22 54 156 4 13 115 136 5 110 161 155 162 1 113 37 120 26 141 47 67 165 30 41 6 131 145 147 159 86 0 56 123 154 15 16 128 132 148 168 63 80 144 20 70 105 133 36 52 66 17 64 103 149 160 46 49 51 71 142 157 27 83 90 96 150 10 59 119 137 170 12 34 99 32 38 68 98 130 24 78 87 108 114 19 44 57 69 74 81 21] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[180 242 200 196 201 260 292 312 172 213 324 336 235 256 301 315 257 314 222 226 305 199 223 240 298 337 192 224 326 268 325 261 267 333 332 230 273 278 295 241 317 184 187 232 239 247 193 216 300 207 307 329 182 203 218 272 282 221 176 179 219 204 233 249 262 188 195 320 339 205 225 269 275 215 280 245 290 197 289 308 331 335 174 246 263 266 294 319 171 191 234 297 211 277 321 303 237 251 206 243 310 175 254 285 227 228 270 299 238 258 318 322 190 250 281 283 306 173 313 183 255 327 212 217 186 189 236 252 309 248 316 330 259 323 194 288 311 177 253 328 276 231 209 279 208 304 244 181 229 220 198 210 293 334 338 287 340 178 264 271 185 286 302 274 202 214 265 284 291 296]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 log_level:Info replication_type:capi active:true checkpoint_interval:1800 doc_batch_size_kb:2048 source_nozzle_per_node:2 stats_interval:1000 filter_expression: worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256]
CheckpointManager 2017-04-25T13:49:03.552-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701...
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:49:03.563-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
StatisticsManager 2017-04-25T13:49:03.563-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:49:03.566-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:49:03.566-05:00 [INFO] ToplogyChangeDetectorSvc has started
StatisticsManager 2017-04-25T13:49:03.566-05:00 [INFO] updateStats started
PipelineRuntimeCtx 2017-04-25T13:49:03.566-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:49:03.566-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:03.566-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:49:03.566-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:49:03.566-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:49:03.566-05:00 [INFO] Checkpointing starts in 1801.92 sec
PipelineRuntimeCtx 2017-04-25T13:49:03.566-05:00 [INFO] Service CheckpointManager has been started
GenericSupervisor 2017-04-25T13:49:03.566-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:49:03.566-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:49:03.566-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:49:03.972-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:03.972-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:03.972-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:49:03.973-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:49:03.974-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:49:03.992-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:49:04.022-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:49:04.022-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:49:04.022-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:49:04.033-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:49:04.033-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:49:04.123-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:49:04.124-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:49:04.125-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:04.125-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:49:04.125-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:49:04.125-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:49:04.125-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:49:04.171-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:49:04.171-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:49:04.171-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:49:04.190-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:49:04.190-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:49:04.343-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:49:04.343-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:04.343-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:49:04.371-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:49:04.371-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:49:04.371-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:49:04.371-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701 is started----------
PipelineManager 2017-04-25T13:49:04.371-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:49:04.371-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:49:04.371-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:49:04.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:49:04.869-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:49:04.869-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:49:04.870-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:49:04.870-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:49:04 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:49:04 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:49:04.993-05:00 [INFO] Send batch count=89
2017/04/25 13:49:05 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:49:05.024-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:05.024-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:05.030-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:49:05.032-05:00 [INFO] Send batch count=77
CapiNozzle 2017-04-25T13:49:05.046-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:05.046-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:49:05 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 112 successful
CapiNozzle 2017-04-25T13:49:05.063-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:49:05.071-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:49:05 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 218 successful
PipelineManager 2017-04-25T13:49:05.211-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:49:05 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 225 successful
CapiNozzle 2017-04-25T13:49:05.296-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:49:05 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 233 successful
CapiNozzle 2017-04-25T13:49:05.320-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:49:05 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 34 successful
CapiNozzle 2017-04-25T13:49:05.366-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:05.377-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:49:05 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 334 successful
CapiNozzle 2017-04-25T13:49:05.435-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:49:05 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 42 successful
CapiNozzle 2017-04-25T13:49:05.454-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:49:05 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 54 successful
CapiNozzle 2017-04-25T13:49:05.535-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:05.561-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
2017/04/25 13:49:05 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 258 successful
CapiNozzle 2017-04-25T13:49:05.618-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:05.618-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:49:05.618-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:49:05.618-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
ReplicationManager 2017-04-25T13:49:05.618-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
StatisticsManager 2017-04-25T13:49:05.628-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
PipelineManager 2017-04-25T13:49:05.632-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:49:05.632-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:49:05.635-05:00 [INFO] err_list=[{"time":"2017-04-25T13:49:05.635600619-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:49:05.635-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:49:05 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 265 successful
CapiNozzle 2017-04-25T13:49:05.725-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:05.725-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:49:05.725-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:49:05.725-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:05.725-05:00 [INFO] Send batch count=500
2017/04/25 13:49:05 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 78 successful
CapiNozzle 2017-04-25T13:49:05.758-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:05.758-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:05.793-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:49:05.799-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:49:05 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 176 successful
CapiNozzle 2017-04-25T13:49:05.811-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:05.828-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:49:05 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 89 successful
CheckpointManager 2017-04-25T13:49:05.853-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
CheckpointManager 2017-04-25T13:49:05.853-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:49:05 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:49:05 UPR_STREAMREQ for vb 97 successful
DcpNozzle 2017-04-25T13:49:05.974-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:49:06.038-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:06.215-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:06.244-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:49:06.417-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:06.440-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
StatisticsManager 2017-04-25T13:49:06.575-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:49:06.575-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701 {"Errors": "[{\"time\":\"2017-04-25T13:49:05.635600619-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 66, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 25306, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 66, "set_filtered": 0, "set_received_from_dcp": 25306, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:49:06.750-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:06.768-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:49:06.987-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:07.005-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:49:07.287-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:07.287-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:49:07.287-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:49:07.287-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:07.287-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:07.439-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:07.458-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:07.472-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:49:07.509-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:07.509-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:49:07.509-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:49:07.509-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:07.509-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:07.621-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:07.621-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:07.644-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:49:07.769-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:07.781-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:49:07.991-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:08.027-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:49:08.124-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:08.149-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:49:08.379-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:08.419-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:49:08.523-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:08.523-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:49:08.523-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:49:08.523-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:08.523-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:08.658-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:08.658-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:08.707-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:49:08.872-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:08.872-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:49:08.872-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:49:08.872-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:08.872-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:08.977-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:08.977-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:08.998-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:49:09.529-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:09.550-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:49:10.035-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:10.048-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:49:10.512-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:10.536-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:49:10.891-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:10.905-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:49:10.905-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:49:10.905-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:10.905-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:11.014-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:11.014-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:11.035-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:49:11.790-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:11.814-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:49:11.971-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:49:12.003-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:49:12.490-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:12.490-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:49:12.490-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:49:12.490-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:12.490-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:12.560-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:12.560-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:12.587-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:49:12.977-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:49:12.988-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
GenericSupervisor 2017-04-25T13:49:13.566-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:49:13.566-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:49:13.566-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:49:13.725-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:49:13.735-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:49:13.978-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:49:13.978-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:49:13.978-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:49:13.978-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:13.978-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:14.077-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:14.077-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:14.088-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
ToplogyChangeDetector 2017-04-25T13:49:14.125-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:49:14.552-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:49:14.563-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:49:14.742-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:49:14.753-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:49:15.411-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:49:15.411-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:49:15.411-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:49:15.411-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:15.411-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:15.503-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:15.504-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:15.514-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
PipelineManager 2017-04-25T13:49:15.635-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:49:15.635-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:49:15.636-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:15.636-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
PipelineRuntimeCtx 2017-04-25T13:49:15.636-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:49:15.636-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:15.636-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:49:15.636-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:15.636140449 -0500 CDT
GenericSupervisor 2017-04-25T13:49:15.636-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:15.636-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:49:15.636-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:15.636-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:49:15.636-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:15.636-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:49:15.636-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:49:15.636-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:49:15.636-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:49:15.636-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-517878701 has been stopped
GenericPipeline 2017-04-25T13:49:15.636-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-517878701 is stopped
PipelineManager 2017-04-25T13:49:15.636-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:49:15.636-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:05.635600619-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:49:15.637-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:15.637-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
GenericPipeline 2017-04-25T13:49:15.637-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
GenericPipeline 2017-04-25T13:49:15.637-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
GenericPipeline 2017-04-25T13:49:15.637-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-517878701
AsyncListener 2017-04-25T13:49:15.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:15.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:15.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:15.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:15.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:15.637-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:15.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:15.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:15.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:15.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:15.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:15.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:49:15.638-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:15.638-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:49:15.638-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:49:15.638-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:15.638-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:49:15.638-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:49:15.638-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:15.638-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:49:15.638-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:15.639-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:49:15.639-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:15.639-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:49:15.639-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is not running, no need to resetConn
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:15.639-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:15.639-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:49:15.639-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:15.638896328 -0500 CDT
DcpNozzle 2017-04-25T13:49:15.639-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:49:15.639-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:15.639171956 -0500 CDT
CapiNozzle 2017-04-25T13:49:15.639-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:15.640-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:49:15.640-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:15.640-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:15.640-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:15.641-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:49:15.641-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:15.641-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:49:15.646-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:49:15 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:49:15 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:49:15.647-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:49:15 Stream Closed for vb 1 StreamEnd simulated
DcpNozzle 2017-04-25T13:49:15.647-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:15.647346111 -0500 CDT
DcpNozzle 2017-04-25T13:49:15.647-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:49:15.647-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:15.647-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:49:15.647-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:15.6476084 -0500 CDT
DcpNozzle 2017-04-25T13:49:15.647-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:49:15.647-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:15.647-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:49:15.659-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:49:15.706-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:49:15.706-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:49:15.706-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:15.706-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
XDCRFactory 2017-04-25T13:49:15.769-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:15.770-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:49:15.770-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:49:15.850-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:15.851-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:49:15.851-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:15.851-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:49:15.851-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:15.851-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:49:15.851-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:49:15.852-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:49:15.852-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
GenericSupervisor 2017-04-25T13:49:15.854-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:49:15.854-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
GenericSupervisor 2017-04-25T13:49:15.854-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:15.854-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:15.854-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:15.854-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:15.854-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:49:15.854-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
CheckpointManager 2017-04-25T13:49:15.854-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
CapiUtils 2017-04-25T13:49:15.907-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:15.907-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:15.907-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:49:15.908-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
StatisticsManager 2017-04-25T13:49:15.909-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:49:15.909-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
XDCRFactory 2017-04-25T13:49:15.909-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:49:15.909-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698 is constructed, start it
GenericPipeline 2017-04-25T13:49:15.910-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[315 189 201 263 266 171 174 202 185 220 243 280 287 330 180 204 223 218 227 248 316 199 321 328 337 198 242 307 238 193 256 261 317 322 244 179 274 301 325 258 285 288 210 217 230 265 278 323 181 213 284 319 228 269 282 289 246 247 309 277 237 255 298 334 216 195 209 294 318 203 279 297 339 332 324 240 222 207 283 264 273 313 219 271 331 212 292 172 232 327 226 241 254 275 320 200 206 236 215 221 304 305 314 336 183 225 233 267 194 296 308 312 329 188 205 259 270 208 310 191 192 235 249 306 302 173 268 333 239 326 197 187 253 272 291 335 177 182 184 190 178 252 260 290 276 340 286 186 214 234 262 293 299 311 338 229 231 245 257 281 295 300 176 211 250 251 196 224 303 175] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[68 80 118 13 58 2 126 135 136 9 60 64 149 17 153 50 82 134 143 113 127 48 91 105 150 100 163 1 21 107 144 3 83 4 154 69 130 42 93 99 156 35 41 51 81 114 52 86 98 138 32 59 34 71 55 104 111 12 24 72 148 14 33 89 167 7 22 124 129 53 120 84 96 112 132 66 79 102 164 168 6 39 110 25 170 19 74 31 157 0 20 146 115 140 47 10 46 61 73 158 57 90 162 169 29 37 43 152 106 133 36 77 95 128 54 88 139 49 56 8 159 116 63 87 160 11 109 44 75 117 18 97 103 108 142 165 28 76 92 137 123 119 155 78 38 147 151 121 145 23 70 45 101 27 131 125 15 30 166 122 161 16 40 65 67 85 94 141 26 62 5]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[286 340 186 214 234 262 293 299 311 338 229 231 245 176 257 281 295 300 211 250 251 175 196 224 303 189 201 315 171 263 266 174 202 180 185 220 243 280 287 330 204 223 218 227 248 316 199 321 328 337 198 242 307 238 193 256 261 317 322 244 179 274 301 325 210 258 285 288 217 230 265 278 323 181 213 284 319 228 269 282 289 246 247 309 277 237 255 298 334 195 216 209 294 318 203 279 297 339 332 240 324 222 207 283 264 273 313 219 271 331 212 292 172 232 327 226 241 254 275 320 200 206 236 215 221 304 305 314 336 183 225 233 267 194 296 308 312 329 188 205 259 270 208 310 191 192 235 249 306 173 302 268 333 197 239 326 187 253 272 291 335 177 182 184 190 178 252 260 276 290] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[15 30 166 16 40 122 161 65 26 62 67 85 94 141 5 68 13 58 80 118 2 9 60 126 135 136 64 149 17 153 50 82 134 143 113 48 91 127 105 150 1 21 100 163 3 83 107 144 4 69 130 154 35 41 42 93 99 156 51 81 52 86 114 32 59 98 138 34 71 12 24 55 104 111 72 148 14 7 22 33 89 167 53 120 124 129 84 96 112 132 6 39 66 79 102 164 168 110 25 170 19 74 0 20 31 157 115 140 146 10 46 47 61 73 158 29 37 57 90 162 169 43 152 36 77 106 133 54 88 95 128 49 56 139 8 159 63 87 116 11 109 160 44 75 18 97 117 28 76 103 108 142 165 92 137 123 119 155 78 38 121 145 147 151 23 70 45 101 27 131 125]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 log_level:Info replication_type:capi active:true worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 filter_expression: checkpoint_interval:1800]
CheckpointManager 2017-04-25T13:49:15.911-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698...
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:15.924-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:49:15.924-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:49:15.924-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:49:15.924-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:15.924-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:49:15.924-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:49:15.924-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:49:15.924-05:00 [INFO] Checkpointing starts in 1802.267 sec
PipelineRuntimeCtx 2017-04-25T13:49:15.924-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:49:15.924-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:49:15.925-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:49:15.925-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:49:15.926-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:49:15.926-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:49:15.927-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:49:15.952-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:49:15.952-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:49:15.953-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:49:15.985-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:49:15.986-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:49:15.996-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:49:16.003-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:49:16.003-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:16.003-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:49:16.003-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:49:16.003-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:49:16.003-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:49:16.017-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:49:16.021-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:16.021-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:16.021-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:49:16.028-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:49:16.028-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:49:16.028-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CheckpointManager 2017-04-25T13:49:16.035-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:49:16.044-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:49:16.044-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:49:16.122-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:49:16.122-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:16.122-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:49:16.123-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:49:16.123-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:49:16.123-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:49:16.123-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698 is started----------
PipelineManager 2017-04-25T13:49:16.123-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:49:16.123-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:49:16.123-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:49:16.173-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:49:16.360-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:49:16.360-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:49:16.360-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:49:16.360-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:49:16 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:49:16.457-05:00 [INFO] Send batch count=99
CapiNozzle 2017-04-25T13:49:16.479-05:00 [INFO] Send batch count=302
CapiNozzle 2017-04-25T13:49:16.536-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:16.536-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:49:16 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:49:16.560-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:49:16 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 125 successful
CapiNozzle 2017-04-25T13:49:16.874-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: connection reset by peer
CapiNozzle 2017-04-25T13:49:16.874-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: connection reset by peer.
CapiNozzle 2017-04-25T13:49:16.890-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:49:16 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:49:16 UPR_STREAMREQ for vb 31 successful
StatisticsManager 2017-04-25T13:49:16.985-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:49:17 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 96 successful
CheckpointManager 2017-04-25T13:49:17.654-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
CheckpointManager 2017-04-25T13:49:17.654-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:49:17 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:49:17 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:49:17.768-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:17.790-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
DcpNozzle 2017-04-25T13:49:17.810-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:17.810-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
DcpNozzle 2017-04-25T13:49:17.840-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:17.854-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
StatisticsManager 2017-04-25T13:49:17.983-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:49:18.452-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:18.466-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:49:18.497-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:18.509-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:49:18.982-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:49:19.011-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:19.011-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:49:19.011-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:49:19.011-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:49:19.011-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
PipelineManager 2017-04-25T13:49:19.022-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:49:19.022-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:49:19.025-05:00 [INFO] err_list=[{"time":"2017-04-25T13:49:19.025578381-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:49:19.025-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:49:19.103-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:19.103-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:19.118-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:49:19.283-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:19.283-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:49:19.283-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:49:19.283-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:19.283-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:19.350-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:19.350-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:19.367-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:49:20.268-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
StatisticsManager 2017-04-25T13:49:20.277-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:49:20.278-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698 {"Errors": "[{\"time\":\"2017-04-25T13:49:19.025578381-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 195, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 115456, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 195, "set_filtered": 0, "set_received_from_dcp": 115456, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:49:20.305-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:20.321-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:49:20.332-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
PipelineManager 2017-04-25T13:49:20.435-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:19.025578381-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:49:20.746-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:20.760-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:49:21.200-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:21.221-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:49:21.721-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:21.721-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:49:21.721-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:49:21.721-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:21.721-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:21.815-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:21.815-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:21.840-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:49:22.409-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:22.409-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:49:22.409-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:49:22.409-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:22.409-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:22.527-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:22.527-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:22.556-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:49:22.817-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:22.835-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:49:23.871-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:23.897-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:49:24.047-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:24.074-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:49:24.755-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:24.766-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:49:24.943-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:24.943-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:49:24.943-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:49:24.943-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:24.943-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:25.054-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:25.054-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:25.064-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:49:25.642-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:25.642-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:49:25.642-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:49:25.642-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:25.642-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:25.771-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:25.771-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:25.782-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
GenericSupervisor 2017-04-25T13:49:25.925-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:49:25.925-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:49:25.925-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:49:25.979-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:25.989-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
ToplogyChangeDetector 2017-04-25T13:49:26.193-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:49:26.518-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:26.529-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:49:26.804-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:26.815-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:49:27.361-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:27.372-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:49:27.555-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:27.555-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:49:27.555-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:49:27.555-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:27.556-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:27.639-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:27.639-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:27.650-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:49:28.085-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:28.085-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:49:28.085-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:49:28.085-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:28.085-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:28.187-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:28.187-05:00 [ERROR] batchUpdateDocs for vb 108 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:28.198-05:00 [INFO] Retrying update docs for vb 108 for the 1th time
CapiNozzle 2017-04-25T13:49:28.508-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:49:28.523-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
PipelineManager 2017-04-25T13:49:29.025-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:49:29.025-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:49:29.025-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:29.026-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
PipelineRuntimeCtx 2017-04-25T13:49:29.026-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:49:29.026-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:29.026-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:49:29.026-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:29.02613403 -0500 CDT
GenericSupervisor 2017-04-25T13:49:29.026-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:29.026-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:49:29.026-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:29.026-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:49:29.026-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:29.026-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:49:29.026-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:49:29.026-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:49:29.026-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:49:29.026-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-851671698 has been stopped
GenericPipeline 2017-04-25T13:49:29.026-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-851671698 is stopped
PipelineManager 2017-04-25T13:49:29.026-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:49:29.026-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:19.025578381-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:49:29.026-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:29.027-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
GenericPipeline 2017-04-25T13:49:29.027-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
GenericPipeline 2017-04-25T13:49:29.027-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
GenericPipeline 2017-04-25T13:49:29.027-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-851671698
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:29.027-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:29.028-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:49:29.028-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:29.028-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:49:29.028-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:29.028-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:49:29.028-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:29.028-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:49:29.028-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:29.028-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:29.028-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:29.028291216 -0500 CDT
DcpNozzle 2017-04-25T13:49:29.028-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:49:29.029-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:29.028614058 -0500 CDT
DcpNozzle 2017-04-25T13:49:29.029-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:49:29.029-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:49:29.029-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:29.030-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:49:29.030-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:29.030-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:49:29.030-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:49:29.030-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.030-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:29.030-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.030-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:29.030-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.030-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:29.031-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:29.031-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:49:29.031-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:29.031-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
2017/04/25 13:49:29 Stream Closed for vb 171 StreamEnd simulated
DcpNozzle 2017-04-25T13:49:29.042-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:29.042-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:29.042075891 -0500 CDT
DcpNozzle 2017-04-25T13:49:29.042-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:49:29.042-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:29.042-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:49:29 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:49:29.043-05:00 [INFO] Ask uprfeed to close
2017/04/25 13:49:29 Stream Closed for vb 1 StreamEnd simulated
DcpNozzle 2017-04-25T13:49:29.043-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:29.0432535 -0500 CDT
DcpNozzle 2017-04-25T13:49:29.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:49:29.043-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:29.043-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
XDCRFactory 2017-04-25T13:49:29.158-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:29.158-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:49:29.158-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:49:29.257-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:29.257-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:49:29.257-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:29.257-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:49:29.257-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:29.257-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:49:29.258-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:49:29.258-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:49:29.258-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
GenericSupervisor 2017-04-25T13:49:29.260-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:49:29.260-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
GenericSupervisor 2017-04-25T13:49:29.260-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:29.260-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:29.260-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:29.260-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:29.260-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:49:29.261-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
CheckpointManager 2017-04-25T13:49:29.261-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
CapiUtils 2017-04-25T13:49:29.307-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:29.307-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:29.307-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:49:29.308-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
StatisticsManager 2017-04-25T13:49:29.308-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:49:29.308-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
XDCRFactory 2017-04-25T13:49:29.308-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:49:29.308-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793 is constructed, start it
GenericPipeline 2017-04-25T13:49:29.309-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[275 334 190 281 286 313 283 330 220 242 276 207 315 321 197 269 322 222 238 273 219 296 218 236 284 317 215 234 235 258 287 208 194 231 251 293 279 221 250 209 256 307 173 206 280 233 260 305 193 202 278 335 312 264 282 328 320 323 324 259 298 205 214 232 252 254 338 185 225 262 294 333 196 229 311 177 189 172 176 267 271 303 211 288 227 309 310 188 265 329 216 316 331 178 182 195 224 270 332 228 297 302 306 201 274 325 326 180 246 277 266 301 319 183 171 295 204 253 300 255 179 230 237 241 272 203 247 340 285 175 239 318 289 200 245 257 249 261 290 291 339 240 244 268 299 217 174 181 213 226 243 337 187 304 327 212 248 184 292 314 191 308 336 199 263 186 192 198 210 223] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[0 80 32 11 34 41 101 131 148 102 123 44 73 55 65 68 72 158 3 94 96 154 77 28 37 64 122 6 1 17 62 138 160 74 92 104 27 132 120 142 49 54 59 125 128 146 56 159 30 51 58 75 95 108 116 81 137 139 66 99 150 16 63 121 167 7 130 136 24 2 119 52 70 114 48 69 109 45 89 133 157 166 103 151 163 4 22 134 87 135 153 31 10 33 129 144 46 50 57 88 165 42 84 115 40 61 97 105 111 12 118 126 47 85 90 19 26 145 5 43 71 124 127 25 141 147 23 53 67 93 168 21 79 112 117 162 100 8 20 35 82 39 60 107 152 14 18 29 76 98 9 13 38 110 149 155 170 15 78 86 106 156 91 161 140 36 164 169 83 113 143]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[22 134 87 135 153 31 10 33 129 144 46 50 57 88 165 42 84 115 40 61 97 105 111 12 118 126 47 85 90 19 26 145 5 43 71 124 127 25 141 147 23 53 67 93 168 21 79 112 117 162 100 8 20 35 82 39 60 107 152 14 18 29 76 98 9 13 38 110 149 155 170 15 78 86 106 156 91 161 140 36 164 169 83 113 143 0 80 32 11 34 41 101 131 148 102 123 44 73 55 65 68 72 158 3 94 96 154 77 28 37 64 122 6 1 17 62 138 160 74 92 104 27 132 120 142 49 54 59 125 128 146 56 30 51 58 75 95 108 116 159 81 137 139 66 99 150 16 63 121 167 7 130 136 24 2 119 52 70 114 48 69 109 45 89 133 157 166 103 151 163 4] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[329 216 316 331 178 182 195 224 270 228 297 302 332 201 274 306 325 326 180 246 277 266 301 319 183 171 295 204 253 300 255 179 230 237 241 272 203 247 340 175 239 285 318 200 245 257 289 249 261 290 291 339 240 244 268 299 174 181 213 217 226 243 187 304 327 337 212 248 184 292 314 191 308 336 199 263 186 192 198 210 223 275 190 281 286 313 334 283 330 220 242 276 207 315 197 269 321 322 222 238 273 219 296 218 236 284 317 215 234 235 258 287 208 194 231 251 293 221 250 279 209 256 307 173 206 280 233 260 305 193 202 278 335 312 264 282 328 320 323 324 259 298 205 214 232 252 254 338 185 225 262 294 333 196 229 311 177 189 172 176 267 271 211 288 303 227 309 310 188 265]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 filter_expression: active:true optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info stats_interval:1000]
CheckpointManager 2017-04-25T13:49:29.310-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793...
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:29.323-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:49:29.323-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:29.323-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:49:29.323-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:49:29.323-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:49:29.323-05:00 [INFO] Checkpointing starts in 1803.057 sec
PipelineRuntimeCtx 2017-04-25T13:49:29.323-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:49:29.323-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:49:29.324-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:49:29.324-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:49:29.325-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:49:29.325-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:49:29.325-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:49:29.325-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:49:29.329-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:49:29.354-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:49:29.354-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:49:29.354-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:49:29.368-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:49:29.368-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:49:29.399-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:49:29.417-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:49:29.427-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:29.427-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:29.427-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:49:29.446-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:49:29.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:49:29.452-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:29.452-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:49:29.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:49:29.452-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:49:29.453-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:49:29.496-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:49:29.496-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:49:29.496-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:49:29.510-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:49:29.510-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:49:29.596-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:49:29.597-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:49:29.597-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:29.597-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:49:29.597-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:49:29.597-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:49:29.597-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:49:29.597-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793 is started----------
PipelineManager 2017-04-25T13:49:29.597-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:49:29.597-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:49:29.597-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:49:29.775-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:49:29.776-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:49:29.776-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:49:29.776-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:49:29 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:49:29.958-05:00 [INFO] Send batch count=147
2017/04/25 13:49:29 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:49:29 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:49:29.968-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:49:29.987-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:29.987-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:29.990-05:00 [INFO] Send batch count=69
CapiNozzle 2017-04-25T13:49:30.008-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:49:30 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:49:30.042-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:30.042-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:30.076-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
2017/04/25 13:49:30 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 217 successful
CapiNozzle 2017-04-25T13:49:30.244-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:30.262-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:49:30 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 325 successful
StatisticsManager 2017-04-25T13:49:30.377-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:49:30 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 34 successful
CapiNozzle 2017-04-25T13:49:30.515-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
2017/04/25 13:49:30 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 340 successful
CapiNozzle 2017-04-25T13:49:30.568-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
2017/04/25 13:49:30 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:49:30.861-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:49:30 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 162 successful
CapiNozzle 2017-04-25T13:49:30.878-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:49:30 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:49:30 UPR_STREAMREQ for vb 270 successful
CapiNozzle 2017-04-25T13:49:31.055-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:31.080-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
2017/04/25 13:49:31 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 99 successful
CheckpointManager 2017-04-25T13:49:31.287-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
CheckpointManager 2017-04-25T13:49:31.287-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:49:31 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:49:31 UPR_STREAMREQ for vb 299 successful
DcpNozzle 2017-04-25T13:49:31.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:31.376-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:31.376-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:49:31.376-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:49:31.376-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:49:31.376-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:49:31.396-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:49:31.396-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:49:31.398-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
PipelineManager 2017-04-25T13:49:31.400-05:00 [INFO] err_list=[{"time":"2017-04-25T13:49:31.400884369-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:49:31.401-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
DcpNozzle 2017-04-25T13:49:31.403-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:31.443-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:31.443-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:49:31.443-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:49:31.443-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:31.443-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:31.480-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:31.480-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:31.528-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:49:31.541-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:31.541-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:31.560-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:49:32.229-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:32.246-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
StatisticsManager 2017-04-25T13:49:32.328-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:49:32.328-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793 {"Errors": "[{\"time\":\"2017-04-25T13:49:31.400884369-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 10, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 90, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 56464, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 90, "set_filtered": 0, "set_received_from_dcp": 56460, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:49:32.538-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:32.559-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:49:33.560-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:33.605-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:49:33.792-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:33.806-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:49:34.566-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:34.566-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:49:34.566-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:49:34.566-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:34.566-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:34.635-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:34.635-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:34.658-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:49:34.982-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:34.982-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:49:34.982-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:49:34.982-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:34.982-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:35.074-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:35.074-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:35.123-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
ReplicationManager 2017-04-25T13:49:35.264-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:49:35.279-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:35.292-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
PipelineManager 2017-04-25T13:49:35.393-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:31.400884369-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:49:35.631-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:35.655-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:49:35.805-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:35.825-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:49:36.234-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:36.253-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:49:37.454-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:37.454-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:49:37.454-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:49:37.454-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:37.454-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:37.565-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:37.566-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:37.588-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:49:37.749-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:37.749-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:49:37.749-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:49:37.749-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:37.750-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:37.852-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:37.857-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:37.871-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:49:37.973-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:37.986-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:49:38.206-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:38.227-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:49:38.376-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:38.394-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:49:38.515-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:38.539-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:49:38.889-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:38.889-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:49:38.889-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:49:38.889-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:38.890-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:38.962-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:38.962-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:38.973-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:49:39.099-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:39.099-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:49:39.099-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:49:39.099-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:39.099-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:39.205-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:39.205-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:39.216-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:49:39.217-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:39.228-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
GenericSupervisor 2017-04-25T13:49:39.324-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:49:39.324-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:49:39.324-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:49:39.514-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:39.524-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
ToplogyChangeDetector 2017-04-25T13:49:39.559-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:49:39.669-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:39.680-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:49:39.858-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:39.869-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:49:40.031-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:40.031-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:49:40.031-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:49:40.031-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:40.031-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:40.137-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:40.137-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:40.148-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:49:40.254-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:40.254-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:49:40.254-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:49:40.254-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:40.254-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:40.318-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:40.318-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:40.329-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:49:40.604-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:40.615-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:49:40.771-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:40.782-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:49:40.958-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:40.969-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:49:41.238-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:41.258-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
PipelineManager 2017-04-25T13:49:41.402-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:49:41.402-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:49:41.402-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:41.402-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
PipelineRuntimeCtx 2017-04-25T13:49:41.402-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:49:41.402-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:41.402-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:49:41.402-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:41.402819554 -0500 CDT
GenericSupervisor 2017-04-25T13:49:41.402-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:41.402-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:49:41.402-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:41.402-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:49:41.402-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:41.403-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:49:41.403-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:49:41.403-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:49:41.403-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:49:41.403-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-258045793 has been stopped
AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-258045793 is stopped
PipelineManager 2017-04-25T13:49:41.403-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-258045793
AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
PipelineManager 2017-04-25T13:49:41.403-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:31.400884369-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:49:41.403-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:49:41.403-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:49:41.403-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:41.403-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:49:41.403-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:49:41.403-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:49:41.403-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:49:41.403-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:49:41.404-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:49:41.404-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:49:41.404-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:41.404-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:41.403747008 -0500 CDT
DcpNozzle 2017-04-25T13:49:41.404-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:41.404-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:41.404032033 -0500 CDT
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:41.404-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.404-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:49:41.404-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:49:41.404-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:41.404-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:49:41.405-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.405-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:41.405-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:41.405-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:49:41.405-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:49:41.405-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:49:41.405-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:49:41.405-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:41.405-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:49:41.405-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:41.406-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:49:41.406-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:41.406-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:41.406761588 -0500 CDT
DcpNozzle 2017-04-25T13:49:41.406-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:49:41.406-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:41.406-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:49:41.407-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:41.407-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:41.407817988 -0500 CDT
DcpNozzle 2017-04-25T13:49:41.407-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:49:41.407-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:41.407-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:49:41 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:49:41 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:49:41 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:49:41 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:49:41 Stream Closed for vb 2 StreamEnd simulated
XDCRFactory 2017-04-25T13:49:41.537-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:41.537-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:49:41.537-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:49:41.602-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:41.603-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:49:41.603-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:41.603-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:49:41.603-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:41.603-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:49:41.603-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:49:41.604-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:49:41.604-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
GenericSupervisor 2017-04-25T13:49:41.606-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:49:41.606-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
GenericSupervisor 2017-04-25T13:49:41.606-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:41.606-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:41.606-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:41.606-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:41.606-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:49:41.606-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
CheckpointManager 2017-04-25T13:49:41.606-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
CapiUtils 2017-04-25T13:49:41.634-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:41.634-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:41.634-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:49:41.635-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
StatisticsManager 2017-04-25T13:49:41.635-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:49:41.635-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
XDCRFactory 2017-04-25T13:49:41.635-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:49:41.636-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489 is constructed, start it
GenericPipeline 2017-04-25T13:49:41.637-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[90 17 154 157 42 115 163 167 18 25 55 138 64 120 81 149 98 122 156 34 39 40 63 0 44 99 13 86 137 8 102 114 67 106 140 62 95 2 130 11 52 82 109 83 125 165 168 132 19 30 53 101 24 45 78 21 76 111 22 88 113 148 60 131 142 145 151 153 169 1 3 46 84 36 75 100 117 141 119 128 32 54 68 89 69 96 116 146 28 29 51 66 47 112 124 103 152 4 58 59 9 65 72 170 134 166 7 50 15 127 133 41 91 38 56 104 135 48 143 16 26 93 105 144 23 37 158 27 73 107 150 97 155 43 71 77 85 70 123 136 20 33 61 110 14 108 126 162 49 92 139 164 6 31 12 74 5 57 79 118 35 121 160 161 94 147 80 10 87 129 159] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[180 273 216 206 213 229 286 298 317 171 252 309 322 257 277 224 270 304 174 196 293 190 200 258 294 231 279 314 219 189 266 271 207 220 250 288 183 214 185 260 253 264 195 242 280 297 203 217 240 251 303 212 218 285 333 175 337 176 312 327 241 296 259 278 302 308 228 282 177 210 245 334 181 232 267 326 222 226 295 234 338 187 191 205 246 173 179 324 272 289 300 335 215 336 255 283 287 186 238 248 223 261 268 340 172 193 199 321 332 221 208 330 329 237 256 291 230 233 276 315 204 306 331 201 244 339 269 325 202 305 328 290 292 311 318 262 188 192 197 227 235 263 209 236 247 211 275 254 281 178 239 184 243 249 274 316 299 301 307 313 182 198 284 319 310 320 323 194 225 265]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[166 134 50 7 127 133 15 91 41 38 104 135 56 143 48 16 93 105 144 26 37 158 23 73 107 150 27 71 77 85 97 155 43 70 33 61 110 123 136 20 108 126 162 14 92 139 164 49 31 6 74 12 57 79 118 5 121 160 161 35 147 94 80 87 129 159 10 90 154 157 17 115 163 167 42 25 55 138 18 120 64 149 81 122 156 98 34 40 39 63 44 99 0 86 137 13 102 114 8 67 140 106 95 62 2 52 82 109 130 11 125 165 83 168 30 53 101 132 19 45 78 24 21 111 76 88 113 148 22 131 142 60 3 46 84 145 151 153 169 1 36 100 117 141 75 54 68 89 119 128 32 69 29 51 66 96 116 146 28 112 124 47 103 152 58 4 59 65 72 170 9] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[329 230 237 256 291 233 276 315 201 204 306 331 244 339 269 325 202 305 328 262 290 292 311 318 188 192 197 235 227 209 236 247 263 211 275 254 178 239 281 184 243 249 274 301 307 313 316 299 182 198 284 319 194 225 265 310 320 323 180 216 273 206 213 229 286 317 298 171 252 309 257 277 322 174 196 224 270 304 190 200 258 293 294 231 279 314 189 219 207 266 271 183 220 250 288 214 185 260 195 242 253 264 217 240 251 280 297 203 303 175 212 218 285 333 337 176 312 327 241 296 259 278 302 308 228 177 210 282 245 181 334 267 326 232 222 226 295 234 338 187 191 246 205 173 179 324 215 272 289 300 335 336 186 238 248 255 283 287 172 193 199 223 261 268 340 321 332 221 208 330]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[stats_interval:1000 replication_type:capi active:true worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 filter_expression: checkpoint_interval:1800 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:49:41.637-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489...
AsyncListener 2017-04-25T13:49:41.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:41.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:41.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:49:41.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:41.644-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:49:41.645-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:49:41.645-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:41.645-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:49:41.645-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:49:41.645-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:49:41.645-05:00 [INFO] Checkpointing starts in 1802.433 sec
PipelineRuntimeCtx 2017-04-25T13:49:41.645-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:49:41.645-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:49:41.645-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:49:41.645-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:49:41.647-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:49:41.647-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:49:41.647-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:49:41.647-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:49:41.650-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:49:41.686-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:49:41.686-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiUtils 2017-04-25T13:49:41.689-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:41.689-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:41.689-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:49:41.689-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:49:41.705-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:49:41.709-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:49:41.709-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
ToplogyChangeDetector 2017-04-25T13:49:41.709-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:49:41.717-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:49:41.725-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:49:41.725-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:41.725-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:49:41.725-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:49:41.725-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:49:41.726-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:49:41.758-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:49:41.758-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:49:41.758-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:49:41.779-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:49:41.779-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:49:41.827-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:49:41.887-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:49:41.887-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:41.887-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:49:41.887-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:49:41.887-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:49:41.887-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:49:41.887-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489 is started----------
PipelineManager 2017-04-25T13:49:41.888-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:49:41.888-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:49:41.888-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:49:42.166-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:49:42.166-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:49:42.166-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:49:42.166-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:49:42 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:49:42.274-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:49:42.274-05:00 [INFO] Send batch count=31
CapiNozzle 2017-04-25T13:49:42.282-05:00 [INFO] Send batch count=248
CapiNozzle 2017-04-25T13:49:42.284-05:00 [ERROR] Received malformed response from tcp connection vb=2. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:49:42.284-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err Received malformed response from tcp connection vb=2. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f200/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:49:42.306-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
2017/04/25 13:49:42 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 107 successful
CapiNozzle 2017-04-25T13:49:42.377-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:42.377-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:42.392-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:42.395-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:49:42 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 211 successful
CapiNozzle 2017-04-25T13:49:42.428-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
2017/04/25 13:49:42 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 219 successful
CapiNozzle 2017-04-25T13:49:42.533-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
2017/04/25 13:49:42 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 25 successful
CapiNozzle 2017-04-25T13:49:42.552-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
2017/04/25 13:49:42 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 139 successful
StatisticsManager 2017-04-25T13:49:42.693-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:49:42 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 46 successful
CapiNozzle 2017-04-25T13:49:42.775-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:42.776-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:49:42.776-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:49:42.776-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
ReplicationManager 2017-04-25T13:49:42.776-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]
CapiNozzle 2017-04-25T13:49:42.777-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
PipelineManager 2017-04-25T13:49:42.795-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:49:42.795-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:49:42.798-05:00 [INFO] err_list=[{"time":"2017-04-25T13:49:42.798780011-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]
PipelineManager 2017-04-25T13:49:42.798-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:49:42.806-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:49:42 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 162 successful
CapiNozzle 2017-04-25T13:49:42.907-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:42.909-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:49:42 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 270 successful
CapiNozzle 2017-04-25T13:49:42.926-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:49:42 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:49:42 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 95 successful
CheckpointManager 2017-04-25T13:49:43.162-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
CheckpointManager 2017-04-25T13:49:43.162-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:49:43 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:49:43 UPR_STREAMREQ for vb 294 successful
CapiNozzle 2017-04-25T13:49:43.211-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:43.222-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
DcpNozzle 2017-04-25T13:49:43.234-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:49:43.290-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:43.381-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:43.395-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:49:43.578-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:43.578-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:49:43.578-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:49:43.578-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:43.578-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:49:43.654-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:49:43.654-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489 {"Errors": "[{\"time\":\"2017-04-25T13:49:42.798780011-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 20, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 120, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 16441, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 120, "set_filtered": 0, "set_received_from_dcp": 16441, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:49:43.694-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:43.694-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:43.711-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:49:43.848-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:43.861-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:49:44.207-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:44.224-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:49:44.290-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:49:44.290-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:49:44.290-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:49:44.290-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:44.290-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:44.354-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:44.354-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:44.375-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:49:44.528-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:44.542-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:49:44.732-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:44.752-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:49:44.923-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:49:44.923-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:49:44.923-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:49:44.923-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:44.923-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:45.011-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:45.011-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:45.023-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:49:45.066-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:45.086-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:49:45.306-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:45.323-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:49:45.481-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:45.481-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:49:45.481-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:49:45.481-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:45.481-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:45.558-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:45.558-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:45.589-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:49:45.621-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:45.638-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:49:46.015-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:46.048-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:49:46.106-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:49:46.106-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:49:46.106-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:49:46.106-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:46.106-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:46.185-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:46.185-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:46.205-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:49:46.449-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:46.464-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:49:46.638-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:49:46.656-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
CapiNozzle 2017-04-25T13:49:46.855-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:46.855-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:49:46.855-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:49:46.855-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:46.855-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:46.952-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:46.952-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:46.974-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:49:47.132-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:49:47.148-05:00 [INFO] Retrying update docs for vb 306 for the 3th time
CapiNozzle 2017-04-25T13:49:47.653-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:47.668-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:49:48.653-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:49:48.653-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 306 after 3 retries
CapiNozzle 2017-04-25T13:49:48.653-05:00 [ERROR] Raise error condition batch update docs failed for vb 306 after 3 retries
GenericSupervisor 2017-04-25T13:49:48.653-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:48.653-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:48.739-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:48.743-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:48.784-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:49:48.844-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:48.855-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:49:49.421-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:49.441-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:49:49.632-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:49:49.632-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:49:49.632-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:49:49.633-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:49.633-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:49.704-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:49.704-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:49.715-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:49:49.867-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:49.880-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:49:50.007-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:50.017-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:49:50.163-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:42.798780011-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries]}
]
CapiNozzle 2017-04-25T13:49:50.214-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:49:50.215-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:49:50.215-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:49:50.215-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:50.215-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:50.286-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:50.286-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:50.297-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:49:50.319-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:50.330-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:49:50.674-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:50.685-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:49:50.892-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:49:50.892-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:49:50.892-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:49:50.892-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:50.892-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:50.954-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:50.954-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:50.964-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:49:51.150-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:51.161-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:49:51.625-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:49:51.637-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
GenericSupervisor 2017-04-25T13:49:51.648-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:49:51.648-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:49:51.648-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:49:51.792-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:49:51.792-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:49:51.792-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:49:51.792-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:51.792-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:49:51.878-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:49:51.885-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:51.885-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:51.896-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:49:51.976-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:49:51.987-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:49:52.354-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:52.364-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:49:52.531-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:49:52.531-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:49:52.531-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:49:52.531-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:52.531-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:52.604-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:52.604-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:52.615-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:49:52.756-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:49:52.767-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
PipelineManager 2017-04-25T13:49:52.799-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries 
PipelineManager 2017-04-25T13:49:52.799-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:49:52.799-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:52.799-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
PipelineRuntimeCtx 2017-04-25T13:49:52.799-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:49:52.799-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:52.799-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:49:52.799-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:52.799163 -0500 CDT
GenericSupervisor 2017-04-25T13:49:52.799-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:52.799-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:49:52.799-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:52.799-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:49:52.799-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:49:52.799-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:49:52.799-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:49:52.799-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:49:52.799-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:49:52.799-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-603779489 has been stopped
GenericPipeline 2017-04-25T13:49:52.799-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-603779489 is stopped
PipelineManager 2017-04-25T13:49:52.799-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:49:52.799-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:42.798780011-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 2 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:49:52.799-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:49:52.799-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
GenericPipeline 2017-04-25T13:49:52.799-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
GenericPipeline 2017-04-25T13:49:52.800-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
GenericPipeline 2017-04-25T13:49:52.800-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-603779489
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:52.800-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:49:52.800-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:52.800-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:49:52.800-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:52.800-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:49:52.800-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:49:52.800-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:49:52.800-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:49:52.800-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:49:52.800-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:49:52.800-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:52.800616368 -0500 CDT
DcpNozzle 2017-04-25T13:49:52.800-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:52.800792737 -0500 CDT
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:49:52.801-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:49:52.801-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:49:52.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:49:52.802-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:52.802-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:49:52.802-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:52.802-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:49:52.803-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:52.803-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:49:52.803-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:52.803758047 -0500 CDT
DcpNozzle 2017-04-25T13:49:52.803-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:49:52.803768581 -0500 CDT
DcpNozzle 2017-04-25T13:49:52.803-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:49:52.803-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:49:52.803-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:52.803-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:49:52.803-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:49:52.803-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:49:52 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 3 StreamEnd simulated
2017/04/25 13:49:52 Stream Closed for vb 4 StreamEnd simulated
XDCRFactory 2017-04-25T13:49:54.870-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:54.870-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:49:54.870-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:49:54.914-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:49:54.915-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:49:54.915-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:54.915-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:49:54.915-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:49:54.915-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:49:54.915-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:49:54.915-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:49:54.916-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
GenericSupervisor 2017-04-25T13:49:54.917-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:49:54.917-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
GenericSupervisor 2017-04-25T13:49:54.917-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:54.917-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:54.918-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:54.918-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:49:54.918-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:49:54.918-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
CheckpointManager 2017-04-25T13:49:54.918-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
CapiUtils 2017-04-25T13:49:54.943-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:54.943-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:54.943-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:49:54.943-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
StatisticsManager 2017-04-25T13:49:54.944-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:49:54.944-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
XDCRFactory 2017-04-25T13:49:54.944-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:49:54.944-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082 is constructed, start it
GenericPipeline 2017-04-25T13:49:54.944-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[64 73 158 1 34 150 155 112 55 96 141 151 163 0 32 69 18 40 38 72 29 148 91 22 74 105 115 35 49 3 54 162 169 5 82 23 31 93 110 75 79 156 25 77 24 33 28 66 125 144 20 86 97 101 154 170 42 99 120 142 2 17 119 133 45 98 65 117 123 143 7 89 138 4 124 9 104 127 136 160 6 116 85 103 108 14 47 139 164 80 137 10 13 37 114 165 167 126 130 30 43 51 62 153 36 50 107 109 63 102 46 26 60 131 129 61 106 56 121 100 149 161 168 48 92 166 68 134 57 84 16 145 111 159 12 58 147 8 78 53 81 128 135 140 39 52 152 76 83 95 118 70 90 94 41 19 21 44 71 122 132 157 11 27 87 88 146 15 59 67 113] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[191 213 230 310 331 311 327 220 227 290 193 202 317 335 210 246 323 326 336 319 222 245 250 266 313 197 269 273 289 291 236 280 285 212 183 279 340 248 255 208 298 300 178 194 295 171 244 288 312 204 201 260 320 216 243 303 252 305 233 242 247 258 263 226 207 211 229 264 318 187 219 238 254 278 176 253 172 186 206 301 234 339 272 308 182 274 205 218 304 203 239 283 315 292 293 307 179 200 316 173 189 267 322 332 286 175 309 228 235 329 174 190 214 282 185 334 271 294 177 181 209 223 276 287 224 231 192 221 217 240 249 262 306 314 195 297 333 232 225 281 325 328 275 284 299 259 261 337 180 184 196 251 277 338 188 237 256 296 302 215 241 268 321 199 257 270 324 330 198 265]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[329 174 190 214 282 185 334 271 294 177 181 209 223 276 287 224 231 192 221 217 240 249 262 306 314 195 297 333 232 225 281 325 328 275 284 299 259 261 337 180 184 196 251 277 338 188 237 256 296 302 215 241 268 321 199 257 270 324 330 198 265 191 213 230 310 331 311 327 220 227 290 193 202 317 335 210 246 323 326 336 319 222 245 250 266 313 197 269 273 289 291 236 280 285 212 183 279 340 248 255 208 298 300 178 194 295 171 244 288 312 204 201 260 320 216 243 303 252 305 233 242 247 258 263 226 207 211 229 264 318 187 219 238 254 278 176 253 172 186 206 301 234 339 272 308 182 274 205 218 304 203 239 283 315 292 293 307 179 200 316 173 189 267 322 332 286 175 309 228 235] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[107 109 63 102 46 26 60 131 129 61 106 56 121 100 149 161 168 48 92 166 68 134 57 84 16 145 111 159 12 58 147 8 78 53 81 128 135 140 39 52 152 76 83 95 118 70 90 94 41 19 21 44 71 122 132 157 11 27 87 88 146 15 59 67 113 64 73 158 1 34 150 155 112 55 96 141 151 163 0 32 69 18 40 38 72 29 148 91 22 74 105 115 35 49 3 54 162 169 5 82 23 31 93 110 75 79 156 25 77 24 33 28 66 125 144 20 86 97 101 154 170 42 99 120 142 2 17 119 133 45 98 65 117 123 143 7 89 138 4 124 9 104 127 136 160 6 116 85 103 108 14 47 139 164 80 137 10 13 37 114 165 167 126 130 30 43 51 62 153 36 50]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 active:true worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:49:54.945-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082...
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:49:54.952-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:49:54.952-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:49:54.952-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:49:54.952-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:49:54.952-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:49:54.952-05:00 [INFO] Checkpointing starts in 1800.476 sec
PipelineRuntimeCtx 2017-04-25T13:49:54.952-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:49:54.952-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:49:54.952-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:49:54.952-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:49:54.961-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:49:54.961-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:49:54.961-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:49:54.962-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:49:54.962-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:49:54.962-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:49:54.962-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:49:54.965-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:49:54.991-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:49:54.991-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:49:54.991-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:49:55.005-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:49:55.005-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:49:55.036-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:49:55.039-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:49:55.051-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:49:55.056-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:49:55.056-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:55.056-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:49:55.056-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:49:55.056-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:49:55.056-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:49:55.085-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:49:55.088-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:49:55.088-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:49:55.116-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:49:55.116-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:49:55.208-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:49:55.312-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:49:55.312-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:49:55.312-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:49:55.313-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:49:55.313-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:49:55.313-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:49:55.313-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082 is started----------
PipelineManager 2017-04-25T13:49:55.313-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:49:55.313-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:49:55.313-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:49:56.091-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:49:57.079-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:49:57.318-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:49:57.318-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:49:57.318-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:49:57.319-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:49:57 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:49:57.391-05:00 [INFO] Send batch count=6
CapiNozzle 2017-04-25T13:49:57.395-05:00 [ERROR] Received malformed response from tcp connection vb=100. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f3/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}

CapiNozzle 2017-04-25T13:49:57.395-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err Received malformed response from tcp connection vb=100. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f3/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}
.
CapiNozzle 2017-04-25T13:49:57.458-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:49:57 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:49:57.505-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:57.535-05:00 [INFO] Send batch count=188
2017/04/25 13:49:57 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 319 successful
CapiNozzle 2017-04-25T13:49:57.649-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:57.658-05:00 [ERROR] Received malformed response from tcp connection vb=0. err=malformed HTTP response "0_009_00776_152_16-3.3F\"},{\"rev\":\"1-14b31e10cbb000000000000002000006\",\"id\":\"Meta_2016_1040_008_00049_128_16-3.3F\"},{\"rev\":\"1-14b31e11a07200000000000002000006\",\"id\":\"Meta_2016_1040_005_00352_048_16-3.3F\"}]\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", num_bytes=204, res_buf={0_009_00776_152_16-3.3F"},{"rev":"1-14b31e10cbb000000000000002000006","id":"Meta_2016_1040_008_00049_128_16-3.3F"},{"rev":"1-14b31e11a07200000000000002000006","id":"Meta_2016_1040_005_00352_048_16-3.3F"}]}

CapiNozzle 2017-04-25T13:49:57.658-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err Received malformed response from tcp connection vb=0. err=malformed HTTP response "0_009_00776_152_16-3.3F\"},{\"rev\":\"1-14b31e10cbb000000000000002000006\",\"id\":\"Meta_2016_1040_008_00049_128_16-3.3F\"},{\"rev\":\"1-14b31e11a07200000000000002000006\",\"id\":\"Meta_2016_1040_005_00352_048_16-3.3F\"}]\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", num_bytes=204, res_buf={0_009_00776_152_16-3.3F"},{"rev":"1-14b31e10cbb000000000000002000006","id":"Meta_2016_1040_008_00049_128_16-3.3F"},{"rev":"1-14b31e11a07200000000000002000006","id":"Meta_2016_1040_005_00352_048_16-3.3F"}]}
.
CapiNozzle 2017-04-25T13:49:57.668-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
2017/04/25 13:49:57 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:49:57.682-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:49:57 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:49:57 UPR_STREAMREQ for vb 141 successful
StatisticsManager 2017-04-25T13:49:58.015-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=8, changes_left=315500
2017/04/25 13:49:58 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 49 successful
CapiNozzle 2017-04-25T13:49:58.096-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:58.126-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
2017/04/25 13:49:58 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 267 successful
CapiNozzle 2017-04-25T13:49:58.249-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:49:58 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 166 successful
CapiNozzle 2017-04-25T13:49:58.269-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:49:58 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 77 successful
CapiNozzle 2017-04-25T13:49:58.367-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:58.388-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
2017/04/25 13:49:58 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 293 successful
CheckpointManager 2017-04-25T13:49:58.540-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
CheckpointManager 2017-04-25T13:49:58.540-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:49:58 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:49:58 UPR_STREAMREQ for vb 299 successful
DcpNozzle 2017-04-25T13:49:58.661-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:49:58.718-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:49:58.850-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:58.877-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:49:59.010-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=8, changes_left=315500
CapiNozzle 2017-04-25T13:49:59.034-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:49:59.034-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:49:59.034-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:49:59.034-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries]
ReplicationManager 2017-04-25T13:49:59.034-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries]
PipelineManager 2017-04-25T13:49:59.048-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:49:59.048-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:49:59.051-05:00 [INFO] err_list=[{"time":"2017-04-25T13:49:59.051582344-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries"}]
PipelineManager 2017-04-25T13:49:59.051-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:49:59.112-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:59.112-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:59.137-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:49:59.242-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:49:59.242-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:49:59.242-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:49:59.242-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:49:59.242-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:49:59.322-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:49:59.322-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:49:59.334-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:49:59.437-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:49:59.448-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:49:59.670-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:49:59.690-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:49:59.848-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:49:59.888-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
StatisticsManager 2017-04-25T13:49:59.978-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:49:59.978-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082 {"Errors": "[{\"time\":\"2017-04-25T13:49:59.051582344-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315500, "data_replicated": 1004, "dcp_datach_length": 0, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 403, "docs_filtered": 0, "docs_opt_repd": 4, "docs_processed": 8, "docs_received_from_dcp": 69250, "docs_rep_queue": 0, "docs_written": 4, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 4, "set_failed_cr_source": 403, "set_filtered": 0, "set_received_from_dcp": 69250, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:50:00.136-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:00.153-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:50:00.368-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:50:00.369-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:50:00.369-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:50:00.369-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:00.369-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:00.462-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:00.462-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:00.476-05:00 [INFO] Retrying update docs for vb 308 for the 1th time
CapiNozzle 2017-04-25T13:50:00.628-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:00.628-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:50:00.628-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:50:00.629-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:00.629-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:00.691-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:00.692-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:00.708-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:50:00.974-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:50:00.993-05:00 [INFO] Retrying update docs for vb 308 for the 2th time
CapiNozzle 2017-04-25T13:50:01.144-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:50:01.166-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:50:01.385-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:50:01.408-05:00 [INFO] Retrying update docs for vb 308 for the 3th time
CapiNozzle 2017-04-25T13:50:01.574-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:50:01.592-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:50:01.763-05:00 [ERROR] batchUpdateDocs for vb 308 failed with err receieved unexpected status code, 500, from update docs request for vb 308
.
CapiNozzle 2017-04-25T13:50:01.763-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 308 after 3 retries
CapiNozzle 2017-04-25T13:50:01.763-05:00 [ERROR] Raise error condition batch update docs failed for vb 308 after 3 retries
GenericSupervisor 2017-04-25T13:50:01.763-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:01.763-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:01.791-05:00 [ERROR] batchGetMeta failed. err=Error parsing return value from _revs_diff query for vbucket 310
CapiNozzle 2017-04-25T13:50:01.870-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:01.870-05:00 [ERROR] batchUpdateDocs for vb 310 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:01.888-05:00 [INFO] Retrying update docs for vb 310 for the 1th time
CapiNozzle 2017-04-25T13:50:01.977-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:50:01.978-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:50:01.978-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:50:01.978-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:01.978-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:02.073-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:02.073-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:02.101-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:50:02.600-05:00 [ERROR] batchUpdateDocs for vb 310 failed with err receieved unexpected status code, 500, from update docs request for vb 310
.
CapiNozzle 2017-04-25T13:50:02.613-05:00 [INFO] Retrying update docs for vb 310 for the 2th time
CapiNozzle 2017-04-25T13:50:02.808-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:50:02.834-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:50:03.485-05:00 [ERROR] batchUpdateDocs for vb 310 failed with err receieved unexpected status code, 500, from update docs request for vb 310
.
CapiNozzle 2017-04-25T13:50:03.502-05:00 [INFO] Retrying update docs for vb 310 for the 3th time
CapiNozzle 2017-04-25T13:50:03.633-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:50:03.651-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:50:04.046-05:00 [ERROR] batchUpdateDocs for vb 310 failed with err receieved unexpected status code, 500, from update docs request for vb 310
.
CapiNozzle 2017-04-25T13:50:04.046-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 310 after 3 retries
CapiNozzle 2017-04-25T13:50:04.046-05:00 [ERROR] Raise error condition batch update docs failed for vb 310 after 3 retries
GenericSupervisor 2017-04-25T13:50:04.046-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:04.046-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:04.132-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:04.132-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:04.150-05:00 [INFO] Retrying update docs for vb 304 for the 1th time
CapiNozzle 2017-04-25T13:50:04.223-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:50:04.223-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:50:04.223-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:50:04.223-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:04.223-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:50:06.073-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:50:06.073-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
GenericSupervisor 2017-04-25T13:50:06.097-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:50:06.097-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:50:06.099-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:50:06.154-05:00 [INFO] Retrying update docs for vb 304 for the 2th time
CapiNozzle 2017-04-25T13:50:06.179-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:06.181-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:06.208-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
PipelineManager 2017-04-25T13:50:06.256-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:59.051582344-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries]}
]
CapiNozzle 2017-04-25T13:50:06.540-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:50:06.560-05:00 [INFO] Retrying update docs for vb 304 for the 3th time
CapiNozzle 2017-04-25T13:50:06.750-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:06.769-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:50:06.924-05:00 [ERROR] batchUpdateDocs for vb 304 failed with err receieved unexpected status code, 500, from update docs request for vb 304
.
CapiNozzle 2017-04-25T13:50:06.924-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 304 after 3 retries
CapiNozzle 2017-04-25T13:50:06.924-05:00 [ERROR] Raise error condition batch update docs failed for vb 304 after 3 retries
GenericSupervisor 2017-04-25T13:50:06.924-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:06.924-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:06.993-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:06.993-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:07.004-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:50:07.152-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:07.163-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:50:07.325-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:50:07.343-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:50:07.608-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:07.608-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:50:07.608-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:50:07.608-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:07.608-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:07.700-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:07.700-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:07.711-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:50:07.776-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:50:07.787-05:00 [INFO] Retrying update docs for vb 204 for the 3th time
CapiNozzle 2017-04-25T13:50:08.050-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:50:08.061-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:50:08.285-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:50:08.285-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 204 after 3 retries
CapiNozzle 2017-04-25T13:50:08.285-05:00 [ERROR] Raise error condition batch update docs failed for vb 204 after 3 retries
GenericSupervisor 2017-04-25T13:50:08.285-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:08.285-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:08.371-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:08.371-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:08.382-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:50:08.585-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:50:08.596-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:50:08.916-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:50:08.927-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
PipelineManager 2017-04-25T13:50:09.052-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries 
PipelineManager 2017-04-25T13:50:09.052-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:50:09.052-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:50:09.052-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
PipelineRuntimeCtx 2017-04-25T13:50:09.052-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:50:09.052-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:09.052-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:50:09.052-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:09.052309161 -0500 CDT
GenericSupervisor 2017-04-25T13:50:09.052-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:09.052-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:50:09.052-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:50:09.052-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:50:09.052-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:50:09.052-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:50:09.052-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:50:09.052-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:50:09.052-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:50:09.052-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-915510082 has been stopped
GenericPipeline 2017-04-25T13:50:09.052-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-915510082 is stopped
PipelineManager 2017-04-25T13:50:09.052-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:50:09.052-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
PipelineManager 2017-04-25T13:50:09.052-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:49:59.051582344-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 204 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:50:09.052-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
PipelineManager 2017-04-25T13:50:09.053-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-915510082
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:50:09.053-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:50:09.053-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:50:09.053-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:09.053-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:50:09.053-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:50:09.053-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:50:09.053-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:09.053-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:50:09.053-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:50:09.054-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:50:09.054-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:09.053440753 -0500 CDT
AsyncListener 2017-04-25T13:50:09.054-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:50:09.054-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:50:09.054-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:09.053947517 -0500 CDT
DcpNozzle 2017-04-25T13:50:09.054-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:50:09.054-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:50:09.055-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:50:09.055-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:50:09.055-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:09.055-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
CapiNozzle 2017-04-25T13:50:09.055-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:50:09.055-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:50:09.055-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:09.055-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:50:09.055-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:50:09.055-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:09.055-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:50:09.056-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:50:09.056-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:09.056770111 -0500 CDT
DcpNozzle 2017-04-25T13:50:09.056-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:50:09.056-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:09.056-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:50:09.057-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:50:09.057-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:09.05792979 -0500 CDT
DcpNozzle 2017-04-25T13:50:09.058-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:50:09.058-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:09.058-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:50:09 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:50:09 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:50:09 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:50:09.163-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:50:09.164-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:50:09.164-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:50:09.307-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:50:09.307-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:50:09.308-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:50:09.308-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:50:09.308-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:50:09.308-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:50:09.308-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:50:09.308-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:50:09.309-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
GenericSupervisor 2017-04-25T13:50:09.310-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:50:09.310-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
GenericSupervisor 2017-04-25T13:50:09.310-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:09.310-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:09.310-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:09.310-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:09.310-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:50:09.310-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
CheckpointManager 2017-04-25T13:50:09.310-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
CapiUtils 2017-04-25T13:50:09.370-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:50:09.370-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:50:09.370-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:50:09.371-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
StatisticsManager 2017-04-25T13:50:09.371-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:50:09.371-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
XDCRFactory 2017-04-25T13:50:09.371-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:50:09.371-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909 is constructed, start it
GenericPipeline 2017-04-25T13:50:09.372-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[110 113 59 70 126 6 8 116 152 167 71 81 163 5 53 37 36 47 125 133 11 101 137 162 13 96 118 165 40 25 43 58 140 15 74 99 105 14 87 22 31 85 128 134 144 103 111 164 28 107 44 83 100 17 42 63 158 32 41 7 79 156 46 115 49 117 136 154 161 16 18 91 142 39 69 92 52 121 130 33 86 114 150 24 123 147 34 78 122 51 153 27 21 10 67 84 109 9 72 127 131 166 23 45 57 60 77 89 159 12 66 120 139 65 95 29 104 138 157 38 73 145 151 132 168 19 61 124 148 55 94 106 169 35 50 119 141 64 68 20 97 143 2 75 90 93 56 149 30 108 26 48 76 102 112 170 3 62 155 160 4 54 88 1 80 82 98 0 135 146 129] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[280 267 197 218 264 266 278 193 214 329 201 220 251 316 337 180 189 237 248 327 195 256 330 268 298 229 239 270 308 299 187 200 296 171 190 230 313 325 332 213 294 333 310 247 318 236 307 321 192 209 273 282 249 275 182 208 224 255 179 244 263 176 177 291 338 227 314 234 300 194 207 286 331 215 277 279 324 243 306 339 181 304 206 272 317 185 217 235 240 262 178 253 261 274 172 173 222 285 293 334 196 252 259 260 283 271 312 199 221 269 281 212 254 287 228 245 174 186 231 302 320 184 202 238 297 303 311 191 232 246 295 340 289 305 322 323 288 309 336 205 210 250 290 188 211 226 265 223 326 183 233 242 292 175 241 284 319 225 301 335 219 276 315 203 204 216 258 198 257 328]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[201 220 251 316 180 189 237 248 327 337 195 256 330 268 298 229 239 270 308 299 187 200 171 190 230 296 313 325 332 213 294 333 247 310 318 236 307 192 209 273 282 321 249 275 182 208 224 255 179 244 263 176 177 291 338 227 314 234 300 194 207 286 215 277 279 324 331 243 306 339 181 304 206 272 317 185 217 235 240 262 178 253 261 172 173 222 274 285 293 334 196 252 259 260 283 271 312 199 221 269 281 212 254 287 228 174 186 231 245 302 320 184 202 238 297 303 311 191 232 246 295 340 289 305 322 323 288 309 336 205 210 250 290 188 211 226 265 223 326 183 233 242 292 175 241 284 319 225 301 335 219 276 315 203 204 216 258 198 257 328 280 197 218 264 266 267 278 193 214 329] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[5 53 37 36 47 125 133 11 101 137 162 13 96 118 165 40 25 43 58 140 15 74 99 105 14 87 22 31 85 128 134 144 103 111 164 28 107 44 83 100 17 42 63 158 32 41 7 79 156 46 115 49 117 136 154 161 16 18 91 142 39 69 92 52 121 130 33 86 114 150 24 123 147 34 78 122 51 153 27 21 10 67 84 109 9 72 127 131 23 45 57 60 77 89 159 166 12 66 120 139 65 95 29 104 138 157 38 73 145 151 132 168 19 61 124 148 55 94 106 169 35 50 119 141 64 68 20 97 143 2 75 90 93 56 149 30 108 26 48 76 102 112 170 3 62 155 160 4 54 88 1 80 82 98 0 135 146 129 110 113 59 70 126 6 8 116 152 167 71 81 163]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info filter_expression: active:true checkpoint_interval:1800 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi]
CheckpointManager 2017-04-25T13:50:09.373-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909...
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:09.381-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
ToplogyChangeDetector 2017-04-25T13:50:09.381-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:50:09.381-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:50:09.381-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:09.381-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:50:09.381-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:50:09.381-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:50:09.381-05:00 [INFO] Checkpointing starts in 1802.543 sec
PipelineRuntimeCtx 2017-04-25T13:50:09.381-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:50:09.381-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:50:09.381-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:50:09.381-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:50:09.382-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:50:09.382-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:50:09.383-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:50:09.400-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:50:09.410-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:50:09.410-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:50:09.410-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:50:09.420-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:50:09.469-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:50:09.493-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:50:09.493-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:50:09.493-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:50:09.497-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:50:09.497-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:50:09.527-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:50:09.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:50:09.576-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:50:09.576-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:50:09.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:50:09.576-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:50:09.577-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:50:09.600-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:50:09.600-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:50:09.600-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:50:09.616-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:50:09.616-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:50:09.679-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:50:09.680-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:50:09.680-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:50:09.680-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:50:09.783-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:50:09.783-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:50:09.783-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:50:09.783-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:50:09.783-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:50:09.783-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:50:09.783-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909 is started----------
PipelineManager 2017-04-25T13:50:09.783-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:50:09.783-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:50:09.783-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:50:09 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:50:09.820-05:00 [INFO] Send batch count=26
CapiNozzle 2017-04-25T13:50:09.863-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:09.863-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:09.879-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:50:09 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:50:09 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:50:09.926-05:00 [INFO] Send batch count=311
2017/04/25 13:50:10 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:50:10.020-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:10.020-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:10.041-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:50:10.104-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:50:10 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 209 successful
CapiNozzle 2017-04-25T13:50:10.126-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:50:10 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 51 successful
StatisticsManager 2017-04-25T13:50:10.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:50:10 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 56 successful
CapiNozzle 2017-04-25T13:50:10.588-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:50:10 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 275 successful
CapiNozzle 2017-04-25T13:50:10.646-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:50:10 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 79 successful
CheckpointManager 2017-04-25T13:50:10.717-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
CheckpointManager 2017-04-25T13:50:10.717-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:50:10.783-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:10.783-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:50:10.783-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:50:10.783-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:50:10.783-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
CapiNozzle 2017-04-25T13:50:10.794-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
PipelineManager 2017-04-25T13:50:10.795-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:50:10.795-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:50:10.798-05:00 [INFO] err_list=[{"time":"2017-04-25T13:50:10.798441234-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:50:10.798-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:50:10 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 295 successful
CapiNozzle 2017-04-25T13:50:10.809-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:50:10 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:50:10 UPR_STREAMREQ for vb 97 successful
DcpNozzle 2017-04-25T13:50:10.884-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:50:10.884-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:50:10.889-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:10.889-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:10.901-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:50:11.122-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:11.136-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:50:11.381-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:50:11.387-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:50:11.387-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909 {"Errors": "[{\"time\":\"2017-04-25T13:50:10.798441234-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 161, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 20700, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 161, "set_filtered": 0, "set_received_from_dcp": 20700, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:50:11.398-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:50:11.852-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:11.852-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:50:11.852-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:50:11.852-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:11.852-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:11.936-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:11.936-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:11.953-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:50:12.330-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:12.345-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:50:12.617-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:50:12.630-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:50:12.873-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:12.873-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:50:12.873-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:50:12.873-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:12.873-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:12.966-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:12.966-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:12.996-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:50:13.253-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:50:13.270-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:50:13.661-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:13.693-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:50:14.097-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:50:14.097-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:50:14.097-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:50:14.097-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:14.097-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:14.173-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:14.173-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:14.189-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:50:14.436-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:14.455-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:50:15.194-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:50:15.205-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:50:15.666-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:15.667-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:50:15.667-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:50:15.667-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:15.667-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:15.777-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:15.777-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:15.809-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:50:16.697-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:50:16.742-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:50:16.946-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:16.957-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:50:17.266-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:50:17.266-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:50:17.266-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:50:17.266-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:17.266-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:17.354-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:17.354-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:17.365-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:50:17.408-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:17.419-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:50:17.856-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:17.866-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:50:18.065-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:18.065-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:50:18.065-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:50:18.065-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:18.065-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:18.139-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:18.140-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:18.150-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:50:18.372-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:18.382-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:50:18.610-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:50:18.624-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:50:19.209-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:19.209-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:50:19.210-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:50:19.210-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:19.210-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:19.312-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:19.312-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:19.323-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
GenericSupervisor 2017-04-25T13:50:19.382-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:50:19.382-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:50:19.382-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:50:19.501-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:50:19.512-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
ToplogyChangeDetector 2017-04-25T13:50:19.552-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:50:20.200-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:50:10.798441234-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:50:20.310-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:50:20.321-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:50:20.491-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:50:20.491-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:50:20.491-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:50:20.491-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:20.491-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:20.601-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:20.601-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:20.612-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
PipelineManager 2017-04-25T13:50:20.798-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:50:20.798-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:50:20.798-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:50:20.798-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
PipelineRuntimeCtx 2017-04-25T13:50:20.798-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:50:20.798-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:50:20.798-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:50:20.798-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:50:20.799-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:50:20.799-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:50:20.799-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:50:20.799-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:50:20.799-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:20.799-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:50:20.799-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:20.799172539 -0500 CDT
GenericSupervisor 2017-04-25T13:50:20.799-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:20.799-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:50:20.799-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-308245909 has been stopped
GenericPipeline 2017-04-25T13:50:20.799-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-308245909 is stopped
PipelineManager 2017-04-25T13:50:20.799-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:50:20.799-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:50:10.798441234-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:50:20.799-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:50:20.800-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
GenericPipeline 2017-04-25T13:50:20.800-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
GenericPipeline 2017-04-25T13:50:20.800-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
GenericPipeline 2017-04-25T13:50:20.800-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-308245909
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:20.800-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:50:20.800-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:20.800-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:50:20.800-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:50:20.801-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:50:20.801-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:20.801-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:50:20.801-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:50:20.801-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:50:20.801-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:20.801-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:20.801-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:50:20.801-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:20.801151909 -0500 CDT
DcpNozzle 2017-04-25T13:50:20.802-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:50:20.802-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:20.801659073 -0500 CDT
CapiNozzle 2017-04-25T13:50:20.802-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:50:20.802-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:50:20.802-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:20.802-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:50:20.803-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:50:20.803-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
GenericPipeline 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x65e930
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:50:20.803-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:50:20.803-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:20.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:50:20.803-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:20.803-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:50:20.804-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:50:20.804-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:20.80452996 -0500 CDT
DcpNozzle 2017-04-25T13:50:20.804-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:50:20.804-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:20.804-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:50:20.805-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:50:20.805-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:20.805157703 -0500 CDT
DcpNozzle 2017-04-25T13:50:20.805-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:50:20.805-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:20.805-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:50:20 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:50:20 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:50:20 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:50:20 Stream Closed for vb 172 StreamEnd simulated
XDCRFactory 2017-04-25T13:50:20.905-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:50:20.906-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:50:20.906-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:50:21.000-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:50:21.001-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:50:21.001-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:50:21.001-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:50:21.001-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:50:21.001-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:50:21.002-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:50:21.002-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:50:21.002-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
GenericSupervisor 2017-04-25T13:50:21.004-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:50:21.004-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
GenericSupervisor 2017-04-25T13:50:21.004-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:21.004-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:21.004-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:21.004-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:21.004-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:50:21.004-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
CheckpointManager 2017-04-25T13:50:21.004-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
CapiUtils 2017-04-25T13:50:21.051-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:50:21.051-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:50:21.051-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:50:21.052-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
StatisticsManager 2017-04-25T13:50:21.053-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:50:21.053-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
XDCRFactory 2017-04-25T13:50:21.053-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:50:21.053-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308 is constructed, start it
GenericPipeline 2017-04-25T13:50:21.054-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[199 223 196 235 244 245 298 263 299 289 291 203 214 221 265 327 331 303 184 172 259 317 225 276 321 209 246 308 330 300 312 306 325 234 249 255 302 332 339 208 236 264 186 195 267 182 286 239 311 212 218 227 207 258 280 326 253 278 305 226 210 228 314 334 219 288 335 229 261 252 310 340 187 200 188 202 220 175 272 285 329 179 197 268 318 316 173 294 296 191 204 257 301 205 266 277 181 279 287 293 185 233 237 254 193 248 180 337 297 333 211 256 269 242 284 324 274 198 240 231 232 282 295 319 336 338 224 230 251 206 260 283 290 313 328 177 216 222 194 292 315 247 273 304 171 322 275 281 323 174 215 238 262 183 307 309 320 250 192 189 201 176 190 270 271 178 217 241 213 243] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[51 97 104 134 68 122 19 85 138 168 110 156 5 6 17 61 75 113 119 26 33 121 129 157 3 39 42 58 83 69 153 159 37 93 34 54 63 32 82 149 22 136 105 152 73 77 16 27 70 78 87 101 112 76 55 143 8 148 57 74 90 118 106 141 169 41 114 130 60 29 108 30 36 52 164 81 142 7 15 35 103 12 124 155 2 99 120 154 107 24 86 158 160 46 49 72 94 100 0 38 167 21 128 147 150 151 11 23 132 28 62 127 59 84 95 135 31 98 47 109 146 170 64 91 117 123 140 20 40 9 1 44 144 18 43 125 126 14 45 66 116 131 89 162 10 25 65 80 133 56 88 67 79 102 145 71 163 50 53 111 137 161 4 165 166 48 115 13 139 92 96]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[261 187 200 252 310 340 175 188 202 220 272 285 268 318 329 179 197 294 296 316 173 191 204 257 266 277 301 205 181 279 287 293 185 233 237 193 248 254 180 337 297 256 269 333 211 242 284 324 274 198 240 232 282 231 336 338 295 319 206 224 230 251 313 328 260 283 290 177 216 222 292 315 194 171 247 273 304 322 238 262 275 281 323 174 215 307 183 250 309 320 192 189 201 190 176 217 241 270 271 178 243 213 196 199 223 298 235 244 245 299 263 289 291 203 214 221 265 327 331 303 184 172 259 225 276 317 321 308 330 209 246 300 312 306 325 249 255 234 236 264 302 332 339 208 182 186 195 267 286 212 218 239 311 227 280 207 258 326 226 253 278 305 228 314 210 334 219 288 335 229] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[12 124 155 154 2 99 120 107 24 86 94 100 158 160 46 49 72 0 38 150 151 167 21 128 147 11 23 132 28 62 127 59 84 95 135 31 98 47 109 146 123 140 170 64 91 117 20 40 9 1 44 144 126 18 43 125 116 131 14 45 66 89 162 80 10 25 65 133 56 88 145 67 79 102 71 137 161 163 50 53 111 4 165 166 48 115 13 139 92 96 134 51 97 104 68 122 168 19 85 138 61 75 110 156 5 6 17 113 119 129 157 26 33 121 58 83 3 39 42 69 153 159 37 93 34 54 63 32 82 149 22 136 105 152 73 77 78 87 16 27 70 101 112 76 55 143 8 148 118 57 74 90 106 141 169 41 114 130 60 29 108 164 30 36 52 81 142 103 7 15 35]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info filter_expression: active:true doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi checkpoint_interval:1800 worker_batch_size:500]
CheckpointManager 2017-04-25T13:50:21.054-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308...
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:21.067-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:50:21.067-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:21.067-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:50:21.067-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:50:21.067-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:50:21.067-05:00 [INFO] Checkpointing starts in 1800.588 sec
PipelineRuntimeCtx 2017-04-25T13:50:21.067-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:50:21.067-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:50:21.068-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:50:21.068-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:50:21.070-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:50:21.070-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:50:21.070-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:50:21.070-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:50:21.070-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:50:21.114-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:50:21.114-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:50:21.115-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:50:21.132-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:50:21.132-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:50:21.135-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:50:21.135-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:50:21.135-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:50:21.141-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:50:21.146-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:50:21.151-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:50:21.156-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:50:21.156-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:50:21.156-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:50:21.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:50:21.157-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:50:21.157-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:50:21.231-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:50:21.231-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:50:21.231-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:50:21.249-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:50:21.250-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:50:21.277-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:50:21.427-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:50:21.427-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:50:21.427-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:50:21.427-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:50:21.427-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:50:21.428-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:50:21.428-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308 is started----------
PipelineManager 2017-04-25T13:50:21.428-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:50:21.428-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:50:21.428-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:50:22.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:50:23.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:50:23.638-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:50:23.638-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:50:23.639-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:50:23.639-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:50:23 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:50:23.743-05:00 [INFO] Send batch count=256
2017/04/25 13:50:23 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:50:23.798-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:50:23.803-05:00 [ERROR] Received malformed response from tcp connection vb=202. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f300/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:50:23.803-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err Received malformed response from tcp connection vb=202. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f300/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:50:23.810-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:23.810-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:50:23 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 3 successful
CapiNozzle 2017-04-25T13:50:23.824-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:50:23.824-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:50:23 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 205 successful
CapiNozzle 2017-04-25T13:50:23.880-05:00 [INFO] Send batch count=496
2017/04/25 13:50:23 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:50:23 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 17 successful
StatisticsManager 2017-04-25T13:50:24.115-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:50:24 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 131 successful
CapiNozzle 2017-04-25T13:50:24.188-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:24.218-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:50:24 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 147 successful
CapiNozzle 2017-04-25T13:50:24.422-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:50:24 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 336 successful
CapiNozzle 2017-04-25T13:50:24.450-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
2017/04/25 13:50:24 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 63 successful
CapiNozzle 2017-04-25T13:50:24.729-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:24.748-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:50:24 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 77 successful
CapiNozzle 2017-04-25T13:50:24.945-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:50:24 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 296 successful
CapiNozzle 2017-04-25T13:50:24.968-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
2017/04/25 13:50:24 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:50:24 UPR_STREAMREQ for vb 92 successful
CheckpointManager 2017-04-25T13:50:25.008-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
CheckpointManager 2017-04-25T13:50:25.008-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:50:25 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:50:25 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:50:25 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:50:25 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:50:25 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:50:25.118-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:25.118-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:50:25.118-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:50:25.118-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:50:25.118-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:50:25.126-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:50:25.126-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:50:25.137-05:00 [INFO] err_list=[{"time":"2017-04-25T13:50:25.137349687-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:50:25.137-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:50:25.138-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
DcpNozzle 2017-04-25T13:50:25.139-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:50:25.159-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:50:25.252-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:25.252-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:25.268-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:50:25.358-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:50:25.385-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:50:25.715-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:25.729-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:50:25.944-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:50:25.944-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:50:25.944-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:50:25.944-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:25.944-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:26.031-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:26.032-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:26.049-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
StatisticsManager 2017-04-25T13:50:26.076-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:50:26.076-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308 {"Errors": "[{\"time\":\"2017-04-25T13:50:25.137349687-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 27, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 418, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 53068, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 418, "set_filtered": 0, "set_received_from_dcp": 53068, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:50:26.093-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:26.143-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:50:26.444-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:50:26.467-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:50:26.648-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:50:26.651-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:50:26.652-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:50:26.652-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:26.652-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:26.765-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:26.765-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:26.793-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:50:27.050-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:50:27.076-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:50:27.181-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:27.216-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:50:27.498-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:50:27.498-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:50:27.498-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:50:27.498-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:27.498-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:27.633-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:27.633-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:27.654-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:50:27.668-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:27.699-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:50:28.015-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:28.026-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:50:28.270-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:50:28.270-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:50:28.270-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:50:28.270-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:28.270-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:28.364-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:28.364-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:28.375-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:50:28.464-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:28.476-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:50:29.764-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:29.860-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:50:29.866-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:50:29.866-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:50:29.878-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:50:29.896-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:50:30.494-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
ToplogyChangeDetector 2017-04-25T13:50:31.736-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:50:31.736-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:50:31.736-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:50:31.736-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:50:31.760-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:50:31.785-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:50:31.796-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:50:32.356-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:50:32.354-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:50:33.015-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:50:33.043-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
PipelineManager 2017-04-25T13:50:35.494-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:50:38.259-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:50:38.259-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:50:38.259-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
PipelineRuntimeCtx 2017-04-25T13:50:38.259-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:50:38.259-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:50:38.259-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:50:38.259-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:50:38.259-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:50:38.278-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:50:38.278-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:50:38.278-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:50:38.278-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:38.278-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:50:38.285-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:38.278902401 -0500 CDT
GenericSupervisor 2017-04-25T13:50:38.314-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:38.314-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:50:38.314-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-2016308 has been stopped
GenericPipeline 2017-04-25T13:50:38.314-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-2016308 is stopped
PipelineManager 2017-04-25T13:50:38.314-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:50:38.314-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:50:25.137349687-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:50:38.314-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:50:38.315-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
GenericPipeline 2017-04-25T13:50:38.315-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
GenericPipeline 2017-04-25T13:50:38.315-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
GenericPipeline 2017-04-25T13:50:38.315-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-2016308
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:38.315-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:38.316-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:38.316-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:50:38.316-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:38.316-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:50:38.316-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:50:38.317-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:50:38.318-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:50:38.318-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:38.318-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:50:38.318-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:50:38.318-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:50:38.318-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:38.318-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:50:38.318-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.318-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:50:38.339-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:50:38.339-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:38.318157866 -0500 CDT
CapiNozzle 2017-04-25T13:50:38.339-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:38.31855572 -0500 CDT
DcpNozzle 2017-04-25T13:50:38.339-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:50:38.339-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:50:38.339-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.339-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:50:38.340-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:50:38.343-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:50:38.357-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:50:38.365-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:50:38.365-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:38.365-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:50:38.366-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:50:38.366-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:38.366-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:50:38.374-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:50:38.374-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:50:38.374-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:38.374383245 -0500 CDT
DcpNozzle 2017-04-25T13:50:38.374-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:50:38.374418455 -0500 CDT
DcpNozzle 2017-04-25T13:50:38.374-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:50:38.374-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:50:38.374-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:38.374-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:50:38.374-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:50:38.374-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:50:38 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:50:38 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:50:38 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:50:38 Stream Closed for vb 2 StreamEnd simulated
XDCRFactory 2017-04-25T13:50:38.641-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:50:38.792-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:50:38.907-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
ReplicationManager 2017-04-25T13:50:39.564-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
XDCRFactory 2017-04-25T13:50:39.680-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:50:39.681-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:50:39.681-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:50:39.681-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:50:39.681-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:50:39.681-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:50:39.681-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:50:39.716-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:50:39.818-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
GenericSupervisor 2017-04-25T13:50:39.908-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:50:39.962-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
GenericSupervisor 2017-04-25T13:50:39.980-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:40.089-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:40.249-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:40.258-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:50:40.258-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:50:40.392-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
CheckpointManager 2017-04-25T13:50:40.415-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
PipelineManager 2017-04-25T13:50:40.530-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:50:25.137349687-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Source nozzles are wired to target nozzles}
]
CapiUtils 2017-04-25T13:50:40.598-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:50:40.598-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:50:40.599-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:50:40.600-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
StatisticsManager 2017-04-25T13:50:40.600-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:50:40.600-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
XDCRFactory 2017-04-25T13:50:40.600-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:50:40.601-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 is constructed, start it
GenericPipeline 2017-04-25T13:50:40.602-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[3 27 131 138 32 61 103 122 147 16 62 158 15 82 110 72 96 118 2 4 119 135 6 9 34 36 59 136 68 154 163 167 7 60 37 87 99 17 25 127 148 52 152 11 80 40 74 56 65 134 168 19 50 69 97 38 33 112 142 155 165 21 89 90 149 0 86 115 166 39 46 71 75 140 49 108 5 42 124 128 24 48 77 79 83 85 14 47 53 51 92 117 133 84 93 143 144 81 44 126 132 78 111 164 58 153 57 63 67 114 123 31 130 160 98 106 8 10 45 129 73 22 94 121 28 157 64 100 116 70 102 151 162 12 107 120 137 35 43 54 145 1 29 101 113 169 88 105 146 159 23 95 104 20 26 76 91 109 139 150 55 141 156 41 13 66 170 30 125 18 161] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[225 235 307 329 270 339 287 311 315 173 174 176 181 260 272 278 283 298 340 207 266 282 300 267 306 206 277 327 335 226 202 214 320 188 189 257 184 231 325 250 172 244 223 179 193 252 312 240 247 263 215 196 303 264 274 334 177 180 183 221 238 248 249 271 296 185 204 308 197 205 293 305 322 265 262 302 222 291 328 234 236 269 330 171 187 190 224 314 213 237 324 211 217 228 319 186 194 220 297 273 281 239 280 295 299 191 243 203 209 212 288 338 195 323 208 256 233 242 210 230 284 290 304 310 318 275 337 313 178 332 245 255 259 292 331 200 261 268 326 227 232 241 246 279 316 321 309 336 201 258 198 216 286 301 317 219 276 192 254 182 253 285 294 175 218 229 289 199 251 333]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[158 16 62 110 15 82 118 72 96 119 135 2 4 34 36 59 136 6 9 163 167 68 154 7 60 99 37 87 127 148 17 25 52 152 11 80 40 74 134 168 56 65 69 97 19 50 38 142 155 165 33 112 21 89 90 149 115 166 0 86 71 75 140 39 46 49 108 124 128 5 42 77 79 83 85 24 48 53 14 47 117 133 51 92 143 144 84 93 81 44 126 132 78 111 164 58 153 67 114 123 57 63 160 31 130 98 106 45 129 8 10 73 121 22 94 28 157 116 64 100 151 162 70 102 120 12 107 137 54 145 35 43 1 113 169 29 101 146 159 88 105 23 95 104 20 91 109 139 150 26 76 156 55 141 41 170 13 66 30 125 18 161 3 27 131 138 103 122 147 32 61] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[287 311 315 173 174 176 181 260 283 298 340 272 278 207 266 282 300 267 306 206 277 327 335 226 202 214 320 188 189 257 325 184 231 250 172 244 223 179 193 252 312 240 247 263 215 303 196 264 274 334 180 183 221 238 177 248 249 271 296 185 204 308 205 293 305 322 197 265 262 302 222 291 328 234 236 269 330 171 187 190 224 314 213 324 237 211 217 228 319 194 220 297 186 273 281 239 280 295 299 191 243 203 209 212 288 338 195 323 256 208 233 242 210 230 284 290 304 310 318 275 337 313 178 332 255 259 292 331 245 268 200 261 326 227 232 241 246 321 279 316 336 309 201 258 216 286 301 317 198 219 276 192 254 182 253 285 294 175 218 229 289 199 251 333 225 235 307 329 270 339]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true worker_batch_size:500 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 log_level:Info filter_expression:]
CheckpointManager 2017-04-25T13:50:40.602-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732...
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:50:40.630-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
StatisticsManager 2017-04-25T13:50:40.630-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:50:40.721-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:50:40.721-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:50:40.721-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:50:40.775-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:50:40.802-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:50:40.801-05:00 [INFO] updateStats started
ToplogyChangeDetector 2017-04-25T13:50:40.869-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:50:40.869-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:50:40.872-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:50:40.873-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:50:40.873-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:50:40.873-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:50:40.944-05:00 [INFO] Checkpointing starts in 1801.44 sec
PipelineRuntimeCtx 2017-04-25T13:50:40.960-05:00 [INFO] Service CheckpointManager has been started
GenericSupervisor 2017-04-25T13:50:40.940-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:50:40.994-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:50:40.995-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:50:41.777-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:50:41.777-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:50:41.777-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:50:42.245-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
GenericSupervisor 2017-04-25T13:50:42.327-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
DcpNozzle 2017-04-25T13:50:42.517-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
GenericSupervisor 2017-04-25T13:50:42.525-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
ToplogyChangeDetector 2017-04-25T13:50:42.551-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:50:44.183-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:50:44.540-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:50:45.375-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:50:47.400-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:50:47.401-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:50:47.401-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:50:47.402-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:50:47.402-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:50:47.403-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:50:47.464-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:50:47.465-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:50:47.465-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:50:47.490-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:50:47.490-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:50:47.525-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:50:47.692-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:50:47.845-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:50:47.845-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:50:47.845-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:50:47.846-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:50:47.846-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:50:47.846-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:50:47.846-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 is started----------
PipelineManager 2017-04-25T13:50:47.846-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:50:47.846-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:50:47.846-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:50:47.850-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:50:48.911-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:50:50.018-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:50:50.035-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:50:50.035-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:50:50.035-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericSupervisor 2017-04-25T13:50:52.062-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:50:52.192-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:50:53.003-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:50:25.137349687-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:50:53.005-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:50:53.005-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:50:53.005-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:50:53.005-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [153 16 62 64 95 99 100 132 167 20 38 42 114 126 129 80 157 159 166 1 22 40 60 66 67 148 7 35 51 59 88 102 46 58 94 156 10 21 26 29 34 53 15 48 89 107 9 68 70 125 147 4 37 77 163 152 168 0 2 54 111 128 149 115 119 137 143 145 47 72 75 158 118 5 17 33 39 82 90 170 19 52 84 103 139 151 31 41 50 134 160 8 87 101 112 124 154 138 144 18 24 28 73 108 116 165 23 98 136 14 55 86 109 150 32 45 71 74 113 81 142 164 25 69 79 93 131 133 3 30 65 76 110 121 140 169 44 57 104 106 122 123 141 6 12 61 78 105 130 49 63 91 97 162 27 127 85 96 161 11 56 155 13 92 146 36 43 83 117 120 135]
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [313 179 246 287 171 174 252 260 262 328 253 274 282 310 338 192 200 229 228 244 254 336 193 199 215 251 295 301 306 177 183 212 208 300 331 232 235 269 273 314 180 198 207 238 286 175 188 202 327 195 234 241 276 281 305 323 325 189 217 268 329 332 340 330 186 201 211 267 271 272 289 178 209 256 279 319 335 245 264 266 296 317 322 337 224 233 257 237 277 333 231 240 292 205 293 316 173 194 278 230 283 290 191 204 223 285 185 242 265 291 247 304 334 187 190 225 311 312 172 196 206 324 219 318 320 258 297 303 307 184 210 236 222 239 250 218 221 243 302 321 203 214 216 326 263 315 249 255 261 284 182 197 248 259 288 309 339 176 181 227 270 275 280 294 298 213 220 226 299 308]
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:50:53.006-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
GenericSupervisor 2017-04-25T13:50:53.058-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:50:53.058-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:50:53.058-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:50:54 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 1 successful
StatisticsManager 2017-04-25T13:50:54.614-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:50:54 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:50:54 UPR_STREAMREQ for vb 203 successful
StatisticsManager 2017-04-25T13:50:55.577-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:50:55.629-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:50:55 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 10 successful
StatisticsManager 2017-04-25T13:50:55.745-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:50:55 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 113 successful
StatisticsManager 2017-04-25T13:50:55.923-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:50:55 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:50:55 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:50:56 UPR_STREAMREQ for vb 233 successful
CapiNozzle 2017-04-25T13:50:56.936-05:00 [INFO] Send batch count=151
2017/04/25 13:50:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:50:58 UPR_STREAMREQ for vb 132 successful
ToplogyChangeDetector 2017-04-25T13:50:58.379-05:00 [INFO] validateSourceTopology...
2017/04/25 13:51:00 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:51:00 UPR_STREAMREQ for vb 133 successful
StatisticsManager 2017-04-25T13:51:00.961-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:51:01 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:51:01 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:51:01 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:51:01 UPR_STREAMREQ for vb 34 successful
GenericSupervisor 2017-04-25T13:51:01.262-05:00 [INFO] About to call function
2017/04/25 13:51:07 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:51:07 UPR_STREAMREQ for vb 135 successful
StatisticsManager 2017-04-25T13:51:07.928-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:50:25.137349687-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 1014, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 516, "docs_rep_queue": 515, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 1, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 516, "size_rep_queue": 537520, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 63, "docs_opt_repd": 0, "docs_rep_queue": 761, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 63, "size_rep_queue": 994412, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 1, "dcp_dispatch_time": {"count": 515, "max": 218396, "mean": 1014.6679611650485, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 912, "expiry_received_from_dcp": 0, "set_received_from_dcp": 912}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 912 items, sent 151 items
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:912]
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 912 items, sent 912 items. streams inactive: [80 157 66 67 159 166 40 60 88 148 35 51 59 46 58 53 94 156 48 89 147 68 70 37 77 163 149 152 168 54 145 137 143 47 72 75 158 82 90 39 139 151 170 52 84 160 41 50 134 154 87 138 144 73 165 98 136 150 55 86 45 71 74 81 142 164 69 79 93 65 76 140 169 44 57 141 61 78 162 49 63 91 97 85 96 161 56 155 92 146 135 36 43 83 99 153 62 64 95 167 38 42]
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [187 190 247 172 196 184 258 297 239 250 243 249 255 261 263 182 197 248 284 339 176 181 259 288 280 294 298 270 275 299 179 246 287 171 174 252 260 262 282 338 192 253 274 254 336 193 199 244 177 183 251 295 269 273 180 198 175 188 238 286 195 241 189 268 276 281 340 186 272 289 178 256 267 271 245 264 266 279 337 257 296 277 240 292 293 173 194 278 290 191 283 185 242 265 285 291]
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:07.930-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
GenericSupervisor 2017-04-25T13:51:07.954-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:51:07.964-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
2017/04/25 13:51:07 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:51:07 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:51:07 UPR_STREAMREQ for vb 337 successful
GenericSupervisor 2017-04-25T13:51:02.979-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:51:07.995-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:51:07.995-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:51:07.995-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:51:07.996-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:51:08.026-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:51:08 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:51:08 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 139 successful
StatisticsManager 2017-04-25T13:51:13.117-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:51:10.320-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:51:13 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:51:13 UPR_STREAMREQ for vb 40 successful
ToplogyChangeDetector 2017-04-25T13:51:13.328-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:51:13.334-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
GenericSupervisor 2017-04-25T13:51:13.361-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:51:14.386-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
GenericSupervisor 2017-04-25T13:51:14.397-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:51:14.397-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:51:14.397-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:51:14 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:51:14 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 247 successful
StatisticsManager 2017-04-25T13:51:15.134-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:50:25.137349687-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 4, "dcp_dispatch_time": 870, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 63, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 912, "docs_rep_queue": 761, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 63, "set_filtered": 0, "set_received_from_dcp": 912, "size_rep_queue": 994412, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 2618}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 63, "docs_opt_repd": 0, "docs_rep_queue": 761, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 63, "size_rep_queue": 994412, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 121, "max": 2620, "mean": 2618.5371900826444, "min": 2618}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 4, "dcp_dispatch_time": {"count": 912, "max": 260295, "mean": 870.9144736842105, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 912, "expiry_received_from_dcp": 0, "set_received_from_dcp": 912}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 912 items, sent 151 items
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:912]
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 912 items, sent 912 items. streams inactive: [46 58 53 94 156 89 48 70 147 68 77 163 54 149 152 168 143 145 75 158 47 72 82 90 84 151 170 52 50 160 154 87 73 144 165 98 86 150 55 71 74 45 164 81 142 79 93 69 65 76 169 44 57 61 78 91 97 162 49 63 161 85 96 155 56 146 92 83 43 64 95 99 153 62 167 80 157 60 66 67 159 166 51 59 88 148]
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [249 255 261 263 182 197 248 284 176 181 259 288 298 270 275 280 294 299 179 287 171 174 252 260 262 192 253 274 282 193 199 254 177 183 251 295 180 198 269 273 175 188 286 195 189 268 276 281 186 178 256 267 271 272 289 264 266 279 257 296 277 292 293 173 194 278 191 283 290 185 265 285 291 187 190 172 196 184 258 297 250]
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:15.135-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:15.136-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:15.136-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:15.136-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:15.136-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
PipelineManager 2017-04-25T13:51:15.190-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:51:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 45 successful
StatisticsManager 2017-04-25T13:51:15.360-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:51:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 262 successful
DcpNozzle 2017-04-25T13:51:15.670-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
DcpNozzle 2017-04-25T13:51:15.670-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
CapiNozzle 2017-04-25T13:51:15.671-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:51:15.755-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:51:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:51:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:51:16 UPR_STREAMREQ for vb 159 successful
StatisticsManager 2017-04-25T13:51:17.065-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:51:17 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:51:16 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 66 successful
StatisticsManager 2017-04-25T13:51:17.454-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:51:17.456-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
ToplogyChangeDetector 2017-04-25T13:51:17.495-05:00 [INFO] validateSourceTopology...
2017/04/25 13:51:17 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 274 successful
CapiNozzle 2017-04-25T13:51:17.560-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:51:17 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 278 successful
CapiNozzle 2017-04-25T13:51:17.898-05:00 [INFO] Send batch count=151
2017/04/25 13:51:17 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 72 successful
StatisticsManager 2017-04-25T13:51:17.967-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:51:17.988-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:51:17.996-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:51:17 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:51:17 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 169 successful
CapiNozzle 2017-04-25T13:51:18.023-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:51:18 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:51:18 UPR_STREAMREQ for vb 286 successful
CapiNozzle 2017-04-25T13:51:19.164-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:51:19.167-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:51:19.167-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:51:19.167-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:51:19.167-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:51:19.167-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
2017/04/25 13:51:19 UPR_STREAMREQ for vb 172 successful
CapiNozzle 2017-04-25T13:51:19.194-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
DcpNozzle 2017-04-25T13:51:19.935-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 2
DcpNozzle 2017-04-25T13:51:20.116-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 2
2017/04/25 13:51:20 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 171 successful
PipelineManager 2017-04-25T13:51:20.099-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:51:20.239-05:00 [INFO] Send batch count=500
2017/04/25 13:51:20 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:51:20 UPR_STREAMREQ for vb 188 successful
PipelineManager 2017-04-25T13:51:20.269-05:00 [INFO] err_list=[{"time":"2017-04-25T13:51:20.269734166-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
StatisticsManager 2017-04-25T13:51:20.827-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
GenericSupervisor 2017-04-25T13:51:21.724-05:00 [INFO] About to call function
2017/04/25 13:51:21 UPR_STREAMREQ for vb 189 successful
CapiNozzle 2017-04-25T13:51:22.031-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:51:21 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:51:22 UPR_STREAMREQ for vb 96 successful
GenericSupervisor 2017-04-25T13:51:22.243-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:51:22.620-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:51:22.496-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
GenericSupervisor 2017-04-25T13:51:22.568-05:00 [INFO] Finish executing 0x681fb0
StatisticsManager 2017-04-25T13:51:23.050-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 {"Errors": "[{\"time\":\"2017-04-25T13:51:20.269734166-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 439, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 129, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 1845, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 129, "set_filtered": 0, "set_received_from_dcp": 1845, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:51:23.296-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:51:23.331-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:51:23.331-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:51:23 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:51:23.438-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:51:23 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 194 successful
DcpNozzle 2017-04-25T13:51:23.543-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:51:23 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 196 successful
CheckpointManager 2017-04-25T13:51:23.646-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
CheckpointManager 2017-04-25T13:51:23.653-05:00 [INFO] Done with SetVBTimestamps
PipelineManager 2017-04-25T13:51:23.624-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:51:20.269734166-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
2017/04/25 13:51:23 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:51:23 UPR_STREAMREQ for vb 198 successful
CapiNozzle 2017-04-25T13:51:23.718-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:51:23.718-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:51:23.718-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:51:23.719-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
DcpNozzle 2017-04-25T13:51:23.748-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:51:23.756-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:51:23.965-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:51:24.221-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:51:25.313-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:51:25.314-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:51:25.336-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:51:25.651-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:51:26.546-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:51:27.169-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
ToplogyChangeDetector 2017-04-25T13:51:27.467-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:51:27.583-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:51:28.760-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:51:28.847-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:51:28.900-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:51:28.907-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:51:28.907-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:51:28.907-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:51:28.910-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:51:29.441-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:51:29.441-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:51:29.441-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:51:29.441-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:51:29.441-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:51:29.503-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:51:29.503-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:51:29.552-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:51:29.552-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:51:29.566-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:51:29.580-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:51:31.118-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:51:31.137-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:51:31.200-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:51:31.216-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:51:31.615-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:51:31.797-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:51:32.591-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
PipelineManager 2017-04-25T13:51:32.580-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:51:32.591-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:51:32.591-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:51:32.610-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
PipelineRuntimeCtx 2017-04-25T13:51:32.650-05:00 [INFO] Pipeline context is stopping...
CapiNozzle 2017-04-25T13:51:32.731-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
ToplogyChangeDetector 2017-04-25T13:51:32.802-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:51:32.930-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:51:33.692-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:51:33.748-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:51:33.003989672 -0500 CDT
GenericSupervisor 2017-04-25T13:51:34.271-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:51:34.271-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:51:34.271-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:51:34.271-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:51:34.271-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:51:34.271-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:51:34.271-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:51:34.272-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:51:34.272-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 has been stopped
GenericPipeline 2017-04-25T13:51:34.272-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-681570732 is stopped
PipelineManager 2017-04-25T13:51:34.272-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:51:34.272-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:51:20.269734166-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:51:34.272-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:51:34.273-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
GenericPipeline 2017-04-25T13:51:34.273-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
GenericPipeline 2017-04-25T13:51:34.273-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
GenericPipeline 2017-04-25T13:51:34.273-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-681570732
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:51:34.273-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:51:34.274-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:51:34.274-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:51:34.274-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:51:34.274-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:51:34.274-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:51:34.274-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:51:34.274-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:51:34.274-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:51:34.274-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:51:34.274-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:51:34.275-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:51:34.274449343 -0500 CDT
DcpNozzle 2017-04-25T13:51:34.275-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:51:34.275-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:51:34.274708454 -0500 CDT
CapiNozzle 2017-04-25T13:51:34.275-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:51:34.275-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:51:34.275-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:51:34.276-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:51:34.276-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:51:34.320-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:51:34.320-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:51:34.321-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:51:34.321-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:51:34.321-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:51:34.321-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:51:34.327-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:51:34.327-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:51:34.327793763 -0500 CDT
DcpNozzle 2017-04-25T13:51:34.327-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:51:34.327-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:51:34.328-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:51:34.327955131 -0500 CDT
GenericPipeline 2017-04-25T13:51:34.328-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:51:34.328-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:51:34.328-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:51:34.328-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:51:34.328-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:51:34 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:51:34 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:51:34 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:51:34 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:51:34 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:51:34.588-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:51:34.589-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:51:34.589-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:51:34.660-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:51:34.661-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:51:34.661-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:51:34.661-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:51:34.661-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:51:34.661-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:51:34.661-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:51:34.661-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:51:34.661-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
GenericSupervisor 2017-04-25T13:51:34.662-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:51:34.663-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
GenericSupervisor 2017-04-25T13:51:34.663-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:51:34.663-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:51:34.663-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:51:34.663-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:51:34.663-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:51:34.663-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
CheckpointManager 2017-04-25T13:51:34.663-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
CapiUtils 2017-04-25T13:51:34.685-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:51:34.685-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:51:34.685-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:51:34.685-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
StatisticsManager 2017-04-25T13:51:34.686-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:51:34.686-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
XDCRFactory 2017-04-25T13:51:34.686-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:51:34.686-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 is constructed, start it
GenericPipeline 2017-04-25T13:51:34.687-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[237 284 177 182 205 283 270 197 202 275 278 224 233 299 279 280 285 315 189 181 257 293 306 192 252 186 227 300 324 218 247 264 210 228 267 333 190 317 294 172 248 262 328 258 234 254 206 207 255 298 325 183 198 201 174 178 229 253 330 204 303 329 331 332 290 221 250 334 219 235 273 211 230 261 292 302 304 314 319 176 249 266 242 287 272 312 327 245 259 260 222 291 184 244 246 277 297 187 196 214 268 271 282 310 313 320 208 336 339 171 231 323 243 276 340 216 238 295 338 193 194 322 335 188 179 274 240 263 265 288 175 199 215 318 301 239 269 195 223 173 220 289 307 308 251 321 200 296 309 326 226 203 286 337 209 241 180 225 236 311 191 212 256 281 305 213 232 217 316 185] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[125 141 62 103 152 170 93 127 92 5 8 51 2 98 83 97 155 156 164 14 31 49 74 4 26 91 151 36 80 136 101 87 148 47 147 23 27 129 65 99 10 73 116 37 72 24 66 32 58 159 6 131 169 38 43 78 71 100 115 118 130 28 68 53 165 33 85 67 88 3 17 132 161 121 124 123 77 94 79 133 0 54 50 134 168 13 59 108 1 20 135 42 102 143 84 138 142 154 48 111 25 70 120 109 45 107 160 166 112 128 44 46 18 29 122 137 16 56 69 126 9 12 144 15 60 145 64 82 81 110 117 146 149 76 105 167 21 95 113 140 30 96 139 39 119 35 163 162 19 41 40 150 63 106 61 114 11 34 158 89 157 55 57 153 22 52 104 7 90 75 86]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[291 184 244 246 277 297 187 196 214 268 282 310 313 320 271 208 336 339 171 231 323 243 276 340 216 238 295 338 193 194 322 335 188 179 274 240 263 265 288 175 199 215 318 301 269 195 223 239 220 289 307 308 173 321 251 200 296 309 226 326 203 286 337 209 241 180 225 236 311 191 212 256 281 213 232 305 217 316 185 284 237 182 205 283 177 270 202 275 278 197 224 233 299 280 285 315 189 279 181 293 306 192 252 257 186 227 300 324 218 247 264 228 267 333 190 210 294 317 248 262 328 172 258 254 234 207 255 298 325 206 183 198 201 174 178 229 253 330 204 303 329 331 332 290 221 250 334 235 273 219 230 261 292 302 304 211 314 319 176 249 266 242 287 272 312 327 245 259 260 222] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[42 102 135 84 138 143 154 48 111 142 25 70 120 45 107 109 166 112 128 160 46 18 29 44 137 16 56 122 126 9 12 69 15 60 144 64 82 145 81 117 146 149 76 105 110 21 95 167 140 30 96 113 139 39 119 35 163 19 41 162 40 63 106 150 114 11 34 61 89 157 158 57 153 22 52 55 7 90 104 75 86 125 141 62 103 152 170 93 127 5 8 92 51 2 98 97 155 156 164 14 31 83 74 4 26 49 151 36 80 91 136 101 87 148 147 23 27 47 65 99 129 10 73 116 37 72 24 32 58 66 6 131 159 38 43 169 78 100 115 118 130 28 68 71 53 165 33 85 88 3 17 67 161 121 124 132 77 94 123 133 0 54 79 50 168 13 59 134 1 20 108]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 worker_batch_size:500 source_nozzle_per_node:2 log_level:Info stats_interval:1000 replication_type:capi active:true doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:51:34.688-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748...
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:51:34.697-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
GenericSupervisor 2017-04-25T13:51:34.697-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:51:34.697-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:51:34.697-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:51:34.697-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:51:34.697-05:00 [INFO] Checkpointing starts in 1804.759 sec
PipelineRuntimeCtx 2017-04-25T13:51:34.697-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:51:34.697-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:51:34.698-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:51:34.698-05:00 [INFO] checkpointing rountine started
CapiUtils 2017-04-25T13:51:34.706-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:51:34.707-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:51:34.707-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:51:34.708-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:51:34.713-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:51:34.713-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:51:34.713-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:51:34.713-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:51:34.715-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:51:34.766-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:51:34.766-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CheckpointManager 2017-04-25T13:51:34.769-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:51:34.795-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
StatisticsManager 2017-04-25T13:51:34.806-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:51:34.817-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:51:34.817-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
ToplogyChangeDetector 2017-04-25T13:51:34.856-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:51:35.075-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:51:35.092-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:51:35.935-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:51:35.404-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
CheckpointManager 2017-04-25T13:51:36.049-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:51:36.050-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:51:36.050-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:51:36.051-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:51:35.306-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:51:37.143-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
GenericSupervisor 2017-04-25T13:51:38.388-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:51:42.013-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
CapiNozzle 2017-04-25T13:51:41.927-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:51:42.040-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:51:42.040-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:51:42.152-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:51:42.292-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:51:45.035-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:51:45.745-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:51:46.494-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:51:48.845-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:51:52.847-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
StatisticsManager 2017-04-25T13:51:52.860-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:51:20.269734166-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:51:52.860-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:51:52.860-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:51:52.860-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:51:52.860-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:51:52.860-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [288 297 188 221 256 257 259 269 319 332 333 223 246 266 268 295 315 301 339 210 219 240 242 265 299 184 233 244 247 281 176 197 226 278 175 196 261 262 291 304 171 182 207 211 249 274 298 323 173 239 275 331 206 306 180 195 224 286 312 229 283 284 326 187 198 205 212 220 232 255 292 174 192 241 302 179 189 199 260 334 178 215 218 310 308 325 234 248 253 267 280 293 177 200 243 250 309 311 194 209 238 263 282 287 203 230 251 305 313 314 181 191 270 279 318 277 290 222 252 258 264 271 276 320 340 316 322 193 202 227 235 285 294 329 321 330 183 190 213 217 236 296 335 201 225 303 231 237 327 336 245 273 300 324 337 172 289 307 317 328 185 186 216 228 254 272 204 208 214 338]
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:52.861-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
GenericSupervisor 2017-04-25T13:51:50.383-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:51:52.865-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:51:52.948-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:51:53.191-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:51:53.203-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:51:53.510-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:51:53.513-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
DcpNozzle 2017-04-25T13:51:53.651-05:00 [INFO] Dcp nozzle is started
PipelineManager 2017-04-25T13:51:53.594-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:51:20.269734166-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={The runtime context is started}
]
GenericPipeline 2017-04-25T13:51:54.565-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
GenericPipeline 2017-04-25T13:51:55.933-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:51:55.934-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 is started----------
PipelineManager 2017-04-25T13:51:55.934-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:51:55.934-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:51:55.934-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
GenericSupervisor 2017-04-25T13:51:56.019-05:00 [INFO] About to call function
2017/04/25 13:51:56 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 102 successful
StatisticsManager 2017-04-25T13:51:56.207-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:51:56.339-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:51:56.340-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:51:20.269734166-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:51:56.340-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:51:56.340-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:51:56.340-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items. streams inactive: [34 98 105 153 15 23 33 164 8 31 133 7 111 161 130 18 74 79 93 107 136 148 32 52 54 120 11 29 35 166 59 140 141 51 169 170 26 38 71 82 114 125 134 6 17 37 122 138 157 67 155 42 83 84 94 116 10 21 30 146 70 87 103 112 128 9 48 69 144 159 22 28 108 89 97 163 132 142 152 14 44 106 143 19 46 63 65 104 117 151 165 5 20 25 167 41 96 158 154 36 39 40 16 77 162 109 118 78 80 95 60 92 119 56 91 131 149 64 86 90 139 145 150 160 24 75 81 58 168 88 126 49 57 72 129 135 147 45 55 115 68 73 85 121 124 4 27 61 127 62 66 76 99 13 43 47 156 113 123 137 110 12 50 53]
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items. streams inactive: [298 323 173 239 275 331 206 306 180 195 224 286 312 229 283 284 326 187 198 205 212 220 232 255 292 174 192 241 179 189 199 260 334 178 215 218 310 293 308 325 234 248 253 267 280 177 243 250 287 309 311 194 209 238 263 282 314 203 230 251 305 313 181 191 270 279 318 276 277 290 222 252 258 264 271 320 340 294 316 322 193 202 227 235 285 329 296 321 330 183 190 213 217 236 335 225 303 231 237 327 336 245 273 324 337 172 289 307 317 272 328 185 186 216 228 254 204 208 214 338 269 288 297 188 221 256 257 259 319 332 333 315 223 246 266 268 295 299 339 210 219 240 242 265 184 233 244 247 281 176 197 226 278 304 175 196 261 262 291 274 171 182 207 211 249]
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:51:56.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
2017/04/25 13:51:56 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 325 successful
StatisticsManager 2017-04-25T13:51:56.935-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:51:56 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:51:56 UPR_STREAMREQ for vb 328 successful
CapiNozzle 2017-04-25T13:51:57.001-05:00 [INFO] Send batch count=125
2017/04/25 13:51:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 232 successful
GenericSupervisor 2017-04-25T13:51:57.047-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:51:57.047-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:51:57.047-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:51:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 143 successful
CapiNozzle 2017-04-25T13:51:57.151-05:00 [INFO] Send batch count=334
2017/04/25 13:51:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 52 successful
PipelineManager 2017-04-25T13:51:57.263-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:51:57 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:51:57 UPR_STREAMREQ for vb 159 successful
CapiNozzle 2017-04-25T13:51:57.313-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:51:57.320-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:51:57.351-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:51:57.352-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:52:01 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 190 successful
CapiNozzle 2017-04-25T13:52:01.494-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:52:01.508-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:52:01 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:52:01 UPR_STREAMREQ for vb 95 successful
CapiNozzle 2017-04-25T13:52:01.524-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CheckpointManager 2017-04-25T13:52:01.566-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
CheckpointManager 2017-04-25T13:52:01.566-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:52:01.673-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:52:02.669-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:52:02.694-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:52:02 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:52:02 UPR_STREAMREQ for vb 195 successful
CapiNozzle 2017-04-25T13:52:02.792-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:52:03 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 97 successful
CapiNozzle 2017-04-25T13:52:03.086-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:52:03 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:52:03 UPR_STREAMREQ for vb 197 successful
CapiNozzle 2017-04-25T13:52:03.422-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:52:04 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:52:04 UPR_STREAMREQ for vb 298 successful
StatisticsManager 2017-04-25T13:52:04.288-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:52:38.652-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:52:38.755-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:38.755-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:52:38.755-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:52:38.755-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:52:38.755-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
CapiNozzle 2017-04-25T13:52:38.774-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:38.829-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
PipelineManager 2017-04-25T13:52:38.923-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:52:38.923-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:52:38.935-05:00 [INFO] err_list=[{"time":"2017-04-25T13:52:38.935557754-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:52:38.937-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:52:39.031-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
StatisticsManager 2017-04-25T13:52:39.638-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
StatisticsManager 2017-04-25T13:52:39.638-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:52:39.641-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 {"Errors": "[{\"time\":\"2017-04-25T13:52:38.935557754-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 213, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 2747, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 213, "set_filtered": 0, "set_received_from_dcp": 2747, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

GenericSupervisor 2017-04-25T13:52:39.683-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:52:39.684-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:52:39.684-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:52:39.692-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:52:39.692-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:52:39.698-05:00 [INFO] Finish executing 0x681fb0
ReplicationManager 2017-04-25T13:52:42.189-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:52:42.592-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:42.592-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:42.800-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:52:44.096-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:52:44.096-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:52:44.096-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:52:44.096-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:44.096-05:00 [INFO] Send batch count=500
ReplicationManager 2017-04-25T13:52:44.740-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:52:44.922-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:44.922-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:44.956-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
PipelineManager 2017-04-25T13:52:45.083-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:52:38.935557754-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:52:45.441-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:52:45.481-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:52:45.835-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:45.869-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:52:46.098-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:52:46.116-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:52:46.160-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:46.181-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:52:46.620-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:52:46.620-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:52:46.620-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:52:46.620-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:46.620-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:46.683-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:46.683-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:52:46.683-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:52:46.683-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:46.683-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:46.692-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:46.692-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:46.707-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:52:46.760-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:46.760-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:46.780-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:52:47.233-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:52:47.248-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:52:47.488-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:47.510-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:52:47.696-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:52:47.710-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:52:47.958-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:47.982-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:52:48.128-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:52:48.128-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:52:48.128-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:52:48.128-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:48.128-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:48.257-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:48.257-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:48.277-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:52:48.418-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:48.418-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:52:48.418-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:52:48.418-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:48.418-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:48.502-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:48.502-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:48.523-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:52:48.691-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:52:48.707-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
PipelineManager 2017-04-25T13:52:48.937-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:52:48.937-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:52:48.937-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:52:48.951-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
PipelineRuntimeCtx 2017-04-25T13:52:48.951-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:52:48.951-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:52:48.951-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:52:48.961-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:52:48.95153128 -0500 CDT
GenericSupervisor 2017-04-25T13:52:48.961-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:52:48.961-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:52:48.961-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:52:48.961-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:52:48.961-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:52:48.961-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:52:48.961-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:52:48.961-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:52:48.961-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:52:48.961-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 has been stopped
GenericPipeline 2017-04-25T13:52:48.964-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
GenericPipeline 2017-04-25T13:52:48.964-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-661407748 is stopped
PipelineManager 2017-04-25T13:52:48.964-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:52:48.964-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
GenericPipeline 2017-04-25T13:52:48.964-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:52:48.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:52:48.964-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:52:48.964-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:52:48.964-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:52:38.935557754-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

CapiNozzle 2017-04-25T13:52:48.964-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
PipelineManager 2017-04-25T13:52:48.964-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:52:48.964-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:52:48.964-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:52:48.964989318 -0500 CDT
GenericPipeline 2017-04-25T13:52:48.965-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:52:48.964-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-661407748
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:52:48.965-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:52:48.965-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:52:48.965-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:52:48.965-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:52:48.965-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
CapiNozzle 2017-04-25T13:52:48.965-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:52:48.965582572 -0500 CDT
DcpNozzle 2017-04-25T13:52:48.965-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:52:48.966-05:00 [INFO] Ask uprfeed to close
CapiNozzle 2017-04-25T13:52:48.966-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
DcpNozzle 2017-04-25T13:52:48.966-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:52:48.966275602 -0500 CDT
CapiNozzle 2017-04-25T13:52:48.966-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
DcpNozzle 2017-04-25T13:52:48.966-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:52:48.966-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:52:48.966-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:52:48.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:52:48.966-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:52:48.966-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:52:48.967-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:52:48.967-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:52:48.967442386 -0500 CDT
DcpNozzle 2017-04-25T13:52:48.967-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:52:48.967-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:52:48.967-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:52:48 Stream Closed for vb 0 StreamEnd simulated
AsyncListener 2017-04-25T13:52:49.474-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:52:49.474-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:52:49.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
PipelineManager 2017-04-25T13:52:50.143-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:52:38.935557754-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:52:50.159-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:52:50.160-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:52:50.160-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:52:50.197-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:52:50.197-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:52:50.197-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:52:50.197-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:52:50.197-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:52:50.197-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:52:50.197-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:52:50.198-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:52:50.199-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
GenericSupervisor 2017-04-25T13:52:50.200-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:52:50.200-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
GenericSupervisor 2017-04-25T13:52:50.200-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:52:50.200-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:52:50.200-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:52:50.200-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:52:50.200-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:52:50.200-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
CheckpointManager 2017-04-25T13:52:50.200-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
CapiUtils 2017-04-25T13:52:50.226-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:52:50.226-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:52:50.226-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:52:50.227-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
StatisticsManager 2017-04-25T13:52:50.227-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:52:50.228-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
XDCRFactory 2017-04-25T13:52:50.228-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:52:50.228-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929 is constructed, start it
GenericPipeline 2017-04-25T13:52:50.229-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[296 254 289 191 184 295 259 321 206 208 326 330 176 248 286 315 209 194 205 199 332 182 187 188 204 242 308 290 314 197 228 177 270 282 252 299 312 230 214 328 337 181 245 241 173 210 212 319 232 219 304 202 281 172 305 280 284 288 225 186 335 190 236 239 247 327 243 267 317 318 175 231 171 240 178 195 274 309 189 277 333 285 331 226 237 265 238 198 227 269 324 297 329 262 283 310 174 264 183 207 250 300 311 193 273 287 220 249 253 307 316 260 320 325 263 217 278 185 192 196 235 276 234 244 224 256 266 291 323 272 302 213 223 179 215 255 293 246 322 294 336 203 211 261 271 301 338 229 258 257 339 201 233 279 303 180 334 251 268 313 218 222 298 306 216 275 292 340 200 221] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[11 27 148 15 22 108 122 88 138 32 77 111 87 145 80 123 169 28 29 167 90 105 164 71 20 100 36 102 116 128 58 110 115 137 106 130 30 39 68 95 66 85 129 146 113 120 46 62 153 53 82 7 78 89 147 96 149 21 26 92 112 24 133 157 160 9 13 59 125 4 43 152 165 17 81 63 118 132 143 0 55 72 109 56 67 126 142 144 99 34 48 159 163 135 150 168 151 156 61 65 107 131 54 91 35 154 6 19 37 83 16 103 119 121 23 33 114 2 25 70 117 12 18 75 140 45 51 60 97 47 50 162 170 84 101 52 161 166 14 98 134 139 41 141 76 136 86 127 1 69 104 49 8 124 57 94 31 79 40 42 155 3 93 158 5 44 64 74 38 10 73]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[221 340 200 289 296 254 191 184 259 295 321 176 206 208 326 330 315 248 286 205 209 194 199 332 182 187 188 204 242 308 290 314 228 197 177 270 282 230 252 299 312 337 214 328 245 181 173 241 210 212 319 232 219 304 202 281 172 305 225 280 284 288 186 335 190 236 239 247 327 231 243 267 317 318 175 171 240 178 195 274 309 333 189 277 265 285 331 226 237 198 238 227 269 324 283 297 329 262 264 310 174 183 207 250 300 311 193 273 287 220 249 253 307 316 260 320 325 263 185 217 278 192 196 235 244 276 234 291 224 256 266 302 323 272 223 213 179 215 255 293 246 322 211 294 336 203 258 261 271 301 338 229 233 257 339 201 180 279 303 313 334 251 268 218 222 298 292 306 216 275] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[10 73 11 27 122 148 15 22 108 88 138 32 77 111 87 145 80 123 169 28 29 167 90 105 164 71 20 100 128 36 102 116 137 58 110 115 106 130 95 30 39 68 66 85 129 146 113 120 46 62 153 53 82 147 7 78 89 96 112 149 21 26 92 24 133 157 125 160 9 13 59 165 4 43 152 17 81 63 118 109 132 143 0 55 72 142 56 67 126 144 99 163 34 48 159 135 150 168 131 151 156 61 65 107 54 91 35 154 83 6 19 37 121 16 103 119 23 33 114 117 2 25 70 12 18 75 97 140 45 51 60 170 47 50 162 84 101 52 161 166 139 14 98 134 41 141 76 136 86 127 1 69 104 49 8 124 57 94 31 79 40 42 155 3 74 93 158 5 44 64 38]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 log_level:Info checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression: active:true]
CheckpointManager 2017-04-25T13:52:50.229-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929...
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:52:50.240-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:52:50.240-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:52:50.240-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:52:50.240-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:52:50.240-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:52:50.240-05:00 [INFO] Checkpointing starts in 1800.116 sec
PipelineRuntimeCtx 2017-04-25T13:52:50.240-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:52:50.240-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:52:50.240-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:52:50.240-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:52:50.255-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:52:50.255-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:52:50.255-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:52:50.256-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:52:50.258-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:52:50.279-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:52:50.280-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CapiNozzle 2017-04-25T13:52:50.286-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:52:50.286-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:52:50.286-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:52:50.304-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:52:50.304-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:52:50.304-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:52:50.320-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:52:50.352-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:52:50.369-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:52:50.759-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:52:50.759-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:52:50.759-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:52:50.759-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:52:50.760-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:52:50.760-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:52:50.760-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:52:50.785-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:52:50.785-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:52:50.796-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:52:50.806-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:52:50.806-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:52:50.986-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:52:50.986-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:52:50.987-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:52:50.987-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:52:51 UPR_STREAMREQ for vb 100 successful
StatisticsManager 2017-04-25T13:52:51.362-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:52:51.396-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:52:51.396-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:52:51.396-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:52:51.396-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:52:51.396-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:52:51.397-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:52:51.397-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929 is started----------
PipelineManager 2017-04-25T13:52:51.397-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:52:51.397-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:52:51.397-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:52:51 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:52:51 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 215 successful
CapiNozzle 2017-04-25T13:52:52.071-05:00 [INFO] Send batch count=49
CapiNozzle 2017-04-25T13:52:52.080-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:52.080-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:52:52 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:52:52.110-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:52:52 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 218 successful
CapiNozzle 2017-04-25T13:52:52.148-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:52:52.164-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f302/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:52:52.164-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f302/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
2017/04/25 13:52:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 32 successful
CapiNozzle 2017-04-25T13:52:52.174-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:52:52 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 331 successful
CapiNozzle 2017-04-25T13:52:52.206-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:52.209-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:52.223-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:52:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 340 successful
CapiNozzle 2017-04-25T13:52:52.318-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
StatisticsManager 2017-04-25T13:52:52.336-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:52:52.338-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:52:52 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 148 successful
CapiNozzle 2017-04-25T13:52:52.452-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:52.452-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:52:52.452-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:52:52.452-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:52:52.452-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
2017/04/25 13:52:52 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 265 successful
PipelineManager 2017-04-25T13:52:52.473-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:52:52.473-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:52:52.476-05:00 [INFO] err_list=[{"time":"2017-04-25T13:52:52.476661285-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:52:52.476-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:52:52 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 74 successful
CapiNozzle 2017-04-25T13:52:52.596-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:52.600-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:52:52 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 172 successful
CapiNozzle 2017-04-25T13:52:52.645-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:52:52 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 83 successful
CapiNozzle 2017-04-25T13:52:52.808-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:52:52 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 179 successful
CapiNozzle 2017-04-25T13:52:52.820-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:52:52 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:52:52 UPR_STREAMREQ for vb 291 successful
CheckpointManager 2017-04-25T13:52:52.919-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
CheckpointManager 2017-04-25T13:52:52.919-05:00 [INFO] Done with SetVBTimestamps
DcpNozzle 2017-04-25T13:52:52.962-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:52:53 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:52:53 UPR_STREAMREQ for vb 198 successful
CapiNozzle 2017-04-25T13:52:53.031-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:53.053-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
DcpNozzle 2017-04-25T13:52:53.105-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:52:53.256-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:52:53.256-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929 {"Errors": "[{\"time\":\"2017-04-25T13:52:52.476661285-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 11, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 24, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 3731, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 24, "set_filtered": 0, "set_received_from_dcp": 3731, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:52:53.303-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:52:53.314-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:52:53.474-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:53.493-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:52:53.719-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:52:53.739-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:52:53.918-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:52:53.919-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:52:53.919-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:52:53.919-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:53.919-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:53.972-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:53.973-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:53.983-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:52:54.089-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:52:54.089-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:52:54.089-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:52:54.089-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:54.089-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:54.172-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:54.172-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:54.188-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:52:54.407-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:52:54.419-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:52:54.648-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:52:54.665-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:52:54.873-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:52:54.884-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:52:55.140-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:52:55.157-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:52:55.396-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:52:55.396-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:52:55.396-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:52:55.396-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:55.396-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:55.534-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:55.534-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:55.548-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:52:55.654-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:52:55.654-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:52:55.654-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:52:55.654-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:55.654-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:55.765-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:55.765-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:55.779-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:52:55.938-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:55.949-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:52:56.341-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:52:56.357-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:52:56.796-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:56.823-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:52:57.135-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:52:57.158-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:52:57.408-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:52:57.408-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:52:57.408-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:52:57.408-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:57.408-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:57.513-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:57.513-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:57.528-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:52:57.817-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:52:57.817-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:52:57.817-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:52:57.817-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:57.817-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:57.943-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:52:57.943-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:52:57.965-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:52:58.180-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:52:58.209-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:52:58.515-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:52:58.537-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:52:58.657-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:52:58.678-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:52:58.966-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:52:58.982-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:52:59.327-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:52:59.373-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:52:59.385-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:52:59.385-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:52:59.382-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:52:59.388-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:52:59.401-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
GenericSupervisor 2017-04-25T13:52:59.401-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:52:59.445-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:52:59.445-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:53:00.353-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:53:00.353-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:53:00.353-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:53:00.556-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:00.556-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:00.556-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:00.556-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:00.577-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:53:00.581-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
ToplogyChangeDetector 2017-04-25T13:53:02.399-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:53:02.402-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:02.414-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:53:02.425-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:53:02.436-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
PipelineManager 2017-04-25T13:53:02.482-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:53:02.482-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:53:02.482-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:53:02.482-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
PipelineRuntimeCtx 2017-04-25T13:53:02.482-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:53:02.482-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:53:02.482-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:53:02.482-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:02.482506799 -0500 CDT
GenericSupervisor 2017-04-25T13:53:02.482-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:53:02.482-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:53:02.482-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:53:02.482-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:53:02.482-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:53:02.482-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:53:02.482-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:53:02.483-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:53:02.483-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:53:02.483-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-197809929 has been stopped
GenericPipeline 2017-04-25T13:53:02.483-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-197809929 is stopped
PipelineManager 2017-04-25T13:53:02.483-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:53:02.483-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:52:52.476661285-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:53:02.483-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:53:02.483-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
GenericPipeline 2017-04-25T13:53:02.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
GenericPipeline 2017-04-25T13:53:02.484-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
GenericPipeline 2017-04-25T13:53:02.484-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-197809929
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:02.484-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:02.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:02.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:02.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:02.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:02.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:02.502-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:53:02.502-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:53:02.502-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:53:02.502-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:53:02.503-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:53:02.503-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:53:02.503-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:53:02.503-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:53:02.503-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:53:02.503-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:53:02.503-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:53:02.503-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:53:02.503-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:53:02.503-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.503-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.503-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:53:02.503-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:53:02.503-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:02.503150572 -0500 CDT
DcpNozzle 2017-04-25T13:53:02.503-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:53:02.503-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:02.503568286 -0500 CDT
CapiNozzle 2017-04-25T13:53:02.504-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:53:02.504-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:53:02.505-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:53:02.505-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:53:02.505-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:53:02.505-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.505-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:53:02.505-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:53:02.505-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:53:02.505-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:53:02.505-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:53:02.510-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:53:02.510-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:53:02.510-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:02.510462893 -0500 CDT
DcpNozzle 2017-04-25T13:53:02.510-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:02.510482936 -0500 CDT
DcpNozzle 2017-04-25T13:53:02.510-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:53:02.510-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:53:02.510-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:53:02.510-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:53:02.510-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:53:02.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:53:02.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
2017/04/25 13:53:02 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:53:02 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:53:02.689-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:53:02.690-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:53:02.709-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:53:02.821-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:53:02.822-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:53:02.822-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:53:02.822-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:53:02.822-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:53:02.822-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:53:02.822-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:53:02.828-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:53:02.839-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
GenericSupervisor 2017-04-25T13:53:02.845-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:53:02.845-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
GenericSupervisor 2017-04-25T13:53:02.845-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:02.845-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:02.845-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:02.845-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:02.845-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:53:02.852-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
CheckpointManager 2017-04-25T13:53:02.852-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
CapiUtils 2017-04-25T13:53:02.898-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:53:02.898-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:53:02.898-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:53:02.914-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
StatisticsManager 2017-04-25T13:53:02.920-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:53:02.920-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
XDCRFactory 2017-04-25T13:53:02.921-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:53:02.921-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 is constructed, start it
GenericPipeline 2017-04-25T13:53:02.928-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[60 157 134 156 120 121 58 30 99 101 152 141 10 11 154 92 111 124 3 7 62 123 86 166 32 49 113 133 16 29 90 27 51 118 164 14 79 170 47 61 143 151 4 12 23 24 103 153 42 95 25 59 98 53 128 63 75 155 35 65 126 20 77 81 5 56 122 9 18 28 54 69 8 19 110 112 150 13 93 15 76 142 159 48 114 115 6 37 39 106 109 162 144 45 50 55 57 78 105 107 129 132 46 146 147 33 71 74 80 83 73 89 70 100 125 130 158 163 64 131 135 165 1 87 136 68 21 22 67 91 94 102 148 17 34 72 82 96 161 0 41 85 97 108 127 139 2 137 31 36 44 149 52 160 26 43 117 140 169 38 88 116 119 145 167 66 84 40 104 138 168] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[192 272 186 249 323 296 298 311 178 225 208 187 241 242 220 180 206 210 213 289 227 229 257 179 222 320 244 288 316 184 217 336 211 212 228 235 254 271 255 291 330 334 173 190 284 335 302 172 209 239 275 295 195 312 332 268 277 299 331 337 219 240 266 301 226 329 191 215 231 218 233 253 258 278 307 245 176 248 282 274 314 333 250 182 196 205 216 247 286 224 181 221 304 252 287 294 315 324 198 262 263 339 197 199 203 230 264 310 183 223 269 290 175 313 317 177 189 321 283 297 265 328 236 171 303 193 214 306 325 260 281 300 327 285 308 322 259 305 270 273 338 188 232 238 276 185 202 246 318 319 243 256 309 326 293 234 279 280 292 340 201 237 261 267 174 194 204 207 251 200]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[15 76 142 159 48 114 115 6 37 39 106 109 162 45 50 55 57 78 105 107 144 129 132 46 146 147 33 71 74 80 83 73 89 70 100 125 130 158 163 64 131 135 165 1 87 136 68 21 22 67 91 94 102 148 17 34 72 82 96 161 0 41 85 97 108 127 139 2 137 31 36 44 149 52 160 26 43 117 140 169 38 88 116 119 145 167 66 84 40 104 138 168 60 157 134 156 120 121 58 30 99 101 152 141 10 11 154 92 111 124 3 7 62 123 86 166 32 49 113 133 16 29 90 27 51 118 164 14 79 170 47 61 143 151 4 12 23 24 103 153 42 95 25 59 98 53 128 63 75 155 35 65 126 20 77 81 5 56 122 9 18 28 54 69 8 19 110 112 150 13 93] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[197 199 203 230 264 310 183 223 269 290 175 313 317 177 189 321 283 297 265 328 236 171 303 193 214 306 260 325 281 300 327 285 308 322 259 305 270 273 338 188 232 238 276 185 202 246 318 319 243 256 309 326 293 234 279 280 292 340 201 237 261 267 174 194 204 207 251 200 192 272 186 249 323 296 298 311 178 225 208 187 241 242 180 206 210 213 220 227 229 257 289 179 222 320 244 288 316 184 217 336 211 212 228 235 254 271 255 291 330 334 173 190 284 335 302 172 209 239 275 295 195 312 268 277 299 331 332 337 219 240 266 301 226 329 191 215 231 218 233 253 258 278 307 245 176 248 282 274 314 333 182 196 205 216 247 250 286 181 221 224 304 252 287 294 315 324 198 262 263 339]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[active:true checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 log_level:Info stats_interval:1000 replication_type:capi filter_expression: worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:53:02.928-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608...
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:02.939-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
StatisticsManager 2017-04-25T13:53:02.939-05:00 [INFO] StatisticsManager Starting...
CapiUtils 2017-04-25T13:53:02.950-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:53:02.950-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:53:02.950-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:53:02.984-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:53:02.984-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:53:02.984-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:53:02.985-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
AsyncListener 2017-04-25T13:53:02.994-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:53:02.994-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:53:02.994-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 failed to stop processing events. Leaving it alone to die. err=Execution timed out
PipelineRuntimeCtx 2017-04-25T13:53:03.060-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:53:03.060-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:53:03.060-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:53:03.060-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:53:03.060-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:53:03.060-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:53:03.060-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:53:03.060-05:00 [INFO] Checkpointing starts in 1800.098 sec
PipelineRuntimeCtx 2017-04-25T13:53:03.060-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:53:03.060-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:53:03.062-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:53:03.062-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:53:03.065-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CheckpointManager 2017-04-25T13:53:03.075-05:00 [INFO] Got the bucket metadata for ckmgr
ToplogyChangeDetector 2017-04-25T13:53:03.082-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:53:03.507-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:53:03.545-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
DcpNozzle 2017-04-25T13:53:03.585-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:53:03.585-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiNozzle 2017-04-25T13:53:03.566-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:53:04.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:53:05.715-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:53:05.743-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:53:05.756-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:53:05.793-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
PipelineManager 2017-04-25T13:53:06.037-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:52:52.476661285-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={The runtime context is started}
]
StatisticsManager 2017-04-25T13:53:06.195-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:53:06.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:53:06.366-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:53:06.375-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:53:06.375-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:53:06.380-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:53:06.380-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:53:06.381-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:53:06.852-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:53:07.211-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CapiNozzle 2017-04-25T13:53:07.217-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
DcpNozzle 2017-04-25T13:53:07.286-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:53:07.286-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
2017/04/25 13:53:07 UPR_STREAMREQ for vb 0 successful
StatisticsManager 2017-04-25T13:53:07.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:53:08.042-05:00 [INFO] Send batch count=500
DcpNozzle 2017-04-25T13:53:08.049-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:53:08.049-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:53:08.049-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:53:08.050-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:53:08.050-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:53:08.051-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:53:08.051-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 is started----------
PipelineManager 2017-04-25T13:53:08.051-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:53:08.051-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:53:08.052-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:53:08 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:53:08.144-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:08.144-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:53:08.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:53:08 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 304 successful
CapiNozzle 2017-04-25T13:53:08.176-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:53:08.210-05:00 [INFO] Send batch count=120
2017/04/25 13:53:08 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 156 successful
CapiNozzle 2017-04-25T13:53:08.632-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:53:08.652-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:53:08 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 89 successful
CheckpointManager 2017-04-25T13:53:08.731-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
CheckpointManager 2017-04-25T13:53:08.734-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:53:08 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:53:08 UPR_STREAMREQ for vb 195 successful
DcpNozzle 2017-04-25T13:53:08.781-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:53:08.950-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
GenericSupervisor 2017-04-25T13:53:13.537-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:53:13.712-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:53:13.723-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
StatisticsManager 2017-04-25T13:53:13.762-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
StatisticsManager 2017-04-25T13:53:13.824-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
StatisticsManager 2017-04-25T13:53:13.825-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:52:52.476661285-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 15, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 370, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 3709, "docs_rep_queue": 3089, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 370, "set_filtered": 0, "set_received_from_dcp": 3709, "size_rep_queue": 4036633, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 33}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 316, "docs_opt_repd": 0, "docs_rep_queue": 1361, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 316, "size_rep_queue": 1835381, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 417, "max": 40, "mean": 38.68585131894484, "min": 37}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 54, "docs_opt_repd": 0, "docs_rep_queue": 1728, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 54, "size_rep_queue": 2201252, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 105, "max": 12, "mean": 11.342857142857143, "min": 11}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 1861, "max": 2008, "mean": 12.828, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 1861, "expiry_received_from_dcp": 0, "set_received_from_dcp": 1861}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 1848, "max": 3849, "mean": 18.953, "min": 6}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 1848, "expiry_received_from_dcp": 0, "set_received_from_dcp": 1848}}

StatisticsManager 2017-04-25T13:53:13.825-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 1861 items, sent 500 items
StatisticsManager 2017-04-25T13:53:13.825-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 1848 items, sent 120 items
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:1848]
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 1848 items, sent 1848 items.
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:1861]
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 1861 items, sent 1861 items.
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:53:13.833-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
ToplogyChangeDetector 2017-04-25T13:53:17.391-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:53:17.391-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:53:17.391-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:53:17.391-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:53:17.391-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:53:17.391-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
GenericSupervisor 2017-04-25T13:53:17.459-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:53:17.459-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:53:17.459-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:53:17.531-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:53:17.531-05:00 [INFO] Send batch count=191
PipelineManager 2017-04-25T13:53:17.543-05:00 [INFO] err_list=[{"time":"2017-04-25T13:53:17.543323525-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:53:17.543-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:53:17.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
StatisticsManager 2017-04-25T13:53:17.829-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:53:17.829-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 {"Errors": "[{\"time\":\"2017-04-25T13:53:17.543323525-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 23, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 370, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 4209, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 370, "set_filtered": 0, "set_received_from_dcp": 4209, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:53:18.988-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:18.988-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:19.030-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:53:19.121-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err read tcp 10.202.98.33:9091: i/o timeout.
CapiNozzle 2017-04-25T13:53:19.354-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:53:19.698-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:19.709-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:53:19.776-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:53:19.787-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:53:20.122-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:20.144-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
PipelineManager 2017-04-25T13:53:20.221-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:53:17.543323525-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]}
]
CapiNozzle 2017-04-25T13:53:20.473-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:53:20.483-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:53:20.800-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:20.800-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:53:20.800-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:53:20.800-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:53:20.800-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:53:20.879-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:20.879-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:20.891-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:53:21.067-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:53:21.067-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:53:21.067-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:53:21.067-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:53:21.067-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:53:21.194-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:21.194-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:21.231-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:53:21.254-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:21.272-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:53:21.605-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:53:21.636-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:53:21.721-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:21.739-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
GenericSupervisor 2017-04-25T13:53:23.571-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:53:23.571-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:53:23.596-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
GenericSupervisor 2017-04-25T13:53:23.582-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:53:23.628-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:53:23.628-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:53:23.628-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:53:23.628-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:53:23.628-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:53:23.636-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:53:25.382-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:25.470-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:25.499-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:53:25.515-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:53:25.515-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:53:25.515-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:53:25.515-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:53:25.515-05:00 [INFO] Send batch count=500
ToplogyChangeDetector 2017-04-25T13:53:25.969-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:53:25.906-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
PipelineManager 2017-04-25T13:53:27.949-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:53:27.962-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:53:28.003-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:53:28.041-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericPipeline 2017-04-25T13:53:28.095-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
CapiNozzle 2017-04-25T13:53:28.153-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
PipelineRuntimeCtx 2017-04-25T13:53:28.295-05:00 [INFO] Pipeline context is stopping...
CapiNozzle 2017-04-25T13:53:30.388-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
GenericSupervisor 2017-04-25T13:53:30.085-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:53:30.779-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:30.667113304 -0500 CDT
CapiNozzle 2017-04-25T13:53:32.378-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
GenericSupervisor 2017-04-25T13:53:30.682-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:53:32.422-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:53:32.422-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:53:32.462-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:53:32.685-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CheckpointManager 2017-04-25T13:53:32.640-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:53:37.257-05:00 [INFO] Exits checkpointing routine.
GenericSupervisor 2017-04-25T13:53:37.034-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
StatisticsManager 2017-04-25T13:53:38.453-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:53:38.453-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:53:38.453-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:53:38.454-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 has been stopped
GenericPipeline 2017-04-25T13:53:38.454-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-822614608 is stopped
PipelineManager 2017-04-25T13:53:38.454-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:53:38.454-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:53:17.543323525-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:53:38.454-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:53:38.486-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
GenericPipeline 2017-04-25T13:53:38.488-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
GenericPipeline 2017-04-25T13:53:38.490-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
GenericPipeline 2017-04-25T13:53:38.490-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-822614608
AsyncListener 2017-04-25T13:53:38.490-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:38.490-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:38.491-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:38.491-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:38.491-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:38.491-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:38.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:38.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:38.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:38.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:53:38.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:53:38.492-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:53:38.492-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:53:38.493-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:53:38.493-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:53:38.494-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:53:38.494-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:53:38.494-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:53:38.494-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:53:38.494-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:53:38.494-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:53:38.494-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:53:38.494-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:53:38.495-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:53:38.495-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:53:38.495-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:38.494377485 -0500 CDT
AsyncListener 2017-04-25T13:53:38.495-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:53:38.495-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:53:38.495063953 -0500 CDT
CapiNozzle 2017-04-25T13:53:38.495-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:53:38.495-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
panic: send on closed channel

goroutine 4706 [running]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Receive(0xc3a16ccd80, 0x8edb60, 0xc3e0c95d10, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:372 +0x749
github.com/couchbase/goxdcr/connector.(*Router).Forward(0xc27db90060, 0x8ed920, 0xc3e0a36c60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/connector/router.go:62 +0x221
github.com/couchbase/goxdcr/parts.(*DcpNozzle).processData(0xc3a3cd1680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:435 +0x1cde
created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:254 +0x63d

goroutine 1 [chan receive, 5 minutes]:
main.main()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/main/main.go:142 +0x14c8

goroutine 5 [syscall, 5 minutes]:
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 9 [IO wait, 5 minutes]:
net.(*pollDesc).Wait(0xc208011020, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011020, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010fc0, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc20802b960)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038120, 0xc2080d2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
bufio.(*Reader).fill(0xc208066420)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Read(0xc208066420, 0xc2080ea001, 0x5ff, 0x5ff, 0xc20800bc80, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:174 +0x26c
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc20801f5e0, 0xc2080ea001, 0x5ff, 0x5ff, 0x2, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x5c
encoding/json.(*Decoder).readValue(0xc2080e4000, 0x415d20, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080e4000, 0x8f07a0, 0xc2080e6020, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc2080e6000, 0xc20801fbe0, 0x0, 0x0)
	/usr/local/go/src/net/rpc/jsonrpc/server.go:66 +0x9c
net/rpc.(*Server).readRequestHeader(0xc2080de000, 0x7ff99340bda0, 0xc2080e6000, 0x0, 0x0, 0xc20801fbe0, 0x40ee00, 0x0, 0x0)
	/usr/local/go/src/net/rpc/server.go:575 +0x97
net/rpc.(*Server).readRequest(0xc2080de000, 0x7ff99340bda0, 0xc2080e6000, 0xc2080e0000, 0xc2080dc060, 0x80ef20, 0x0, 0x0, 0x0, 0x0, ...)
	/usr/local/go/src/net/rpc/server.go:542 +0xa2
net/rpc.(*Server).ServeCodec(0xc2080de000, 0x7ff99340bda0, 0xc2080e6000)
	/usr/local/go/src/net/rpc/server.go:461 +0x6d
github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc2080be480, 0xc208095f90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:146 +0x6c8
github.com/couchbase/cbauth/revrpc.BabysitService(0xc208095f90, 0xc2080be480, 0x7ff99340a528, 0xc20801f540, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:242 +0x80
github.com/couchbase/cbauth.runRPCForSvc(0xc2080be480, 0xc2080be4b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:56 +0x193
github.com/couchbase/cbauth.func·004()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:63 +0x36
created by github.com/couchbase/cbauth.startDefault
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/default.go:64 +0x1c9

goroutine 34 [IO wait, 5 minutes]:
net.(*pollDesc).Wait(0xc2080f87d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080f87d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080f8770, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc20802bf10)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f6070, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc2080f6070, 0xc208158058, 0xc2080c3000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080e9c40, 0xc2080c3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e6600)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080e6600, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080e6600, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080d6ab0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080d6ab0, 0xc2080a6600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080debc0, 0xc2080a6600, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080debc0, 0xc2080a6600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080dec00, 0xc2080a6600, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080fa000, 0xc2080de940, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080fa000, 0x80f160, 0xc2080decc0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 12 [select, 5 minutes]:
net/http.(*persistConn).readLoop(0xc208077ce0)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 13 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc208077ce0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 21 [syscall, 5 minutes]:
syscall.Syscall(0x0, 0x0, 0xc2080f3000, 0x1000, 0xc20803b1d0, 0xc208032320, 0x5e)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x0, 0xc2080f3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x0, 0xc2080f3000, 0x1000, 0x1000, 0xeb0f40, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
os.(*File).read(0xc208038000, 0xc2080f3000, 0x1000, 0x1000, 0xc20803b1d0, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:191 +0x5e
os.(*File).Read(0xc208038000, 0xc2080f3000, 0x1000, 0x1000, 0xc208156ff0, 0x0, 0x0)
	/usr/local/go/src/os/file.go:95 +0x91
bufio.(*Reader).fill(0xc208066f00)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadByte(0xc208066f00, 0xa79fd0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:196 +0x86
github.com/couchbase/goxdcr/replication_manager.pollStdin()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:631 +0xaa
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:114 +0xb8

goroutine 4113 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2fc0, 0x12c, 0xc28e33a100, 0xec, 0x100, 0xc36a4c9000, 0xbb, 0x100, 0xc36dab81e0, 0xc36dab8120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 22 [semacquire]:
sync.(*WaitGroup).Wait(0xc2080e9ac0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc20814e1c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:191 +0x428
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 23 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc20814e1d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 24 [runnable]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).HeartBeat_async(0xc208066eb8, 0xc3e0cd2b40, 0xed0919231, 0xc314df819c, 0xeb0f40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:176 +0x1b5
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats(0xeb1a20, 0xc2080e9ae0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:214 +0x549
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xeb1a20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:192 +0x443
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230

goroutine 25 [select, 5 minutes]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xeb1a30)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 26 [select, 5 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc2080545e0, 0x11, 0xc208147fc0, 0xc208066960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc2080545e0, 0x11, 0xc208147fc0, 0xc208066960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc2080545e0, 0x11, 0xc208147fc0, 0xc208066960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc2081569b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 27 [select, 5 minutes]:
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc20802b2d0, 0xc20802ba10, 0xf, 0xc208146fc0, 0xc208066960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x8bb
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc20802b2d0, 0xc20802ba10, 0xf, 0xc208146fc0, 0xc208066960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:244 +0x80
github.com/couchbase/cbauth/metakv.RunObserveChildren(0xc20802ba10, 0xf, 0xc208146fc0, 0xc208066960, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:367 +0x64
github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).observeChildren(0xc208156b40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:66 +0xad
created by github.com/couchbase/goxdcr/replication_manager.(*MetakvChangeListener).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/metakv_change_listener.go:58 +0x73

goroutine 28 [select]:
net/http.(*persistConn).roundTrip(0xc208158370, 0xc3e0742ef0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc3e0413ee0, 0xc3e074c3e0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3e0413ee0, 0x7ff993409fe8, 0xe94600, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3e0413ee0, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3e0413ee0, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3e0413ee0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3e0413ee0, 0x7ff99340dae0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3e062d030, 0x9bbca0, 0x6, 0x7ff99340dae0, 0xc3e074c3a0, 0x80f820, 0xc3e0b3f7b8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc3e0b3f7a0, 0x9bbca0, 0x6, 0x80f820, 0xc3e0b3f7b8, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc3e074c380, 0x15, 0x7ff99340dae0, 0xc3e074c3a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/goxdcr/utils.LocalPool(0xc3e0742ea0, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:85 +0x263
github.com/couchbase/goxdcr/utils.LocalBucketUUID(0xc3e0742ea0, 0xe, 0xc20802ac78, 0x8, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:370 +0x73
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateExistingReplicationSpec(0xc2080f8070, 0xc3e0706c00, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:464 +0x12d
github.com/couchbase/goxdcr/metadata_svc.(*ReplicationSpecService).ValidateAndGC(0xc2080f8070, 0xc3e0706c00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/replication_spec_service.go:524 +0x40
github.com/couchbase/goxdcr/pipeline_manager.CheckPipelines()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:220 +0xfb
github.com/couchbase/goxdcr/replication_manager.(*replicationManager).checkReplicationStatus(0xeb1a20, 0xc208010700)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:209 +0x152
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:137 +0x287

goroutine 29 [select]:
github.com/couchbase/goxdcr/replication_manager.(*Adminport).Start(0xc208066ea0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:102 +0x75b
created by github.com/couchbase/goxdcr/replication_manager.func·001
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/replication_manager.go:141 +0x2de

goroutine 16 [select, 5 minutes]:
net/http.(*persistConn).readLoop(0xc208158000)
	/usr/local/go/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 33 [select, 5 minutes]:
net/http.(*persistConn).writeLoop(0xc208158000)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 30 [runnable]:
github.com/couchbase/goxdcr/log.(*CommonLogger).logMsgf(0xc20802b330, 0x2, 0x9ec7d0, 0x8, 0xa283d0, 0x15, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:135
github.com/couchbase/goxdcr/log.(*CommonLogger).Debugf(0xc20802b330, 0xa283d0, 0x15, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:152 +0x76
github.com/couchbase/goxdcr/replication_manager.(*Adminport).handleRequest(0xc208066ea0, 0xc3e0127c70, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:150 +0x85
github.com/couchbase/goxdcr/replication_manager.(*Adminport).processRequest(0xc208066ea0, 0xc3e09e0580, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:136 +0x180
github.com/couchbase/goxdcr/replication_manager.*Adminport.(github.com/couchbase/goxdcr/replication_manager.processRequest)·fm(0xc3e09e0580, 0x1, 0x1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/replication_manager/adminport.go:68 +0x57
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc208066eb8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:98 +0x943
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 31 [IO wait]:
net.(*pollDesc).Wait(0xc2080108b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080108b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208010850, 0x0, 0x7ff993409df0, 0xc3e023e1d0)
	/usr/local/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc2080380d0, 0xc20814de38, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc2080380d0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc2080670e0, 0x7ff99340d860, 0xc2080380d0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:1728 +0x92
github.com/couchbase/goxdcr/adminport.func·001()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:95 +0x157
created by github.com/couchbase/goxdcr/adminport.(*httpServer).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:99 +0x144

goroutine 35 [IO wait, 5 minutes]:
net.(*pollDesc).Wait(0xc2080115d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080115d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011570, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc2080d5920)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038178, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc208038178, 0xc208077d38, 0xc2080d3000, 0x1000, 0x1000, 0x54fb5b, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801fec0, 0xc2080d3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080667e0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080667e0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:295 +0x257
net/http/internal.readLine(0xc2080667e0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:110 +0x5a
net/http/internal.(*chunkedReader).beginChunk(0xc2080d6ba0)
	/usr/local/go/src/net/http/internal/chunked.go:47 +0x46
net/http/internal.(*chunkedReader).Read(0xc2080d6ba0, 0xc2080eac00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/internal/chunked.go:77 +0xbb
net/http.(*body).readLocked(0xc2080ded00, 0xc2080eac00, 0x600, 0x600, 0xffffffff, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:584 +0x7a
net/http.(*body).Read(0xc2080ded00, 0xc2080eac00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transfer.go:579 +0x115
net/http.(*bodyEOFSignal).Read(0xc2080ded40, 0xc2080eac00, 0x600, 0x600, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1193 +0x285
encoding/json.(*Decoder).readValue(0xc2080fa180, 0xc2080de800, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:124 +0x5e1
encoding/json.(*Decoder).Decode(0xc2080fa180, 0x80f160, 0xc2080dee00, 0x0, 0x0)
	/usr/local/go/src/encoding/json/stream.go:44 +0x7b
github.com/couchbase/cbauth/metakv.func·003()
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:267 +0x134
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:275 +0x46a

goroutine 3324 [IO wait]:
net.(*pollDesc).Wait(0xc39a663330, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc39a663330, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc39a6632d0, 0xc39a850000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc32944f8a8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39a79c060, 0xc39a850000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc39a79c060, 0xc39a67f868, 0xc39a850000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc39a818320, 0xc39a850000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc39a82b440)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc39a82b440, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc39a67f810)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4882 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16ccd80, 0x12c, 0xc210c25400, 0xec, 0x100, 0xc3d41fd000, 0x99, 0x100, 0xc3d4bf4300, 0xc3d4bf4240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 46 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc208077a20)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 53 [IO wait]:
net.(*pollDesc).Wait(0xc2080f8370, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080f8370, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080f8310, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3e001c260)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f60d8, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc2080f60d8, 0xc208158268, 0xc20800f000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080fe6a0, 0xc20800f000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e6b40)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e6b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208158210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 54 [select]:
net/http.(*persistConn).writeLoop(0xc208158210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 45 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc2080118e0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080118e0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011880, 0xc20812c000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3b77b0fb8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038388, 0xc20812c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc208038388, 0xc208077a78, 0xc20812c000, 0x1000, 0x1000, 0x4184af, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080e8b60, 0xc20812c000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208067e00)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208067e00, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208077a20)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 1011 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5ad80, 0x65, 0xc28e33b100, 0xec, 0x100, 0xc28f989000, 0xc6, 0x100, 0xc2904d7bc0, 0xc2904d7b00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1292 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c7a2000, 0x12c, 0xc2b1aac500, 0xec, 0x100, 0xc2b16e7800, 0xc4, 0x100, 0xc2b24c5140, 0xc2b24c5080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4939 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01b60, 0x1dcd6500, 0xc3a179d480, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2c0320, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2c0320)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4941 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01be0, 0x1dcd6500, 0xc3a179d300, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2df040, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2df040)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1704 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0xc9, 0xc2fd4b5000, 0xec, 0x100, 0xc2fd9d0000, 0xbd, 0x100, 0xc2fda111a0, 0xc2fda110e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4957 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e12420)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 765 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6b40, 0x67, 0xc210c24700, 0xec, 0x100, 0xc26c19d000, 0xb3, 0x100, 0xc26c5c27e0, 0xc26c5c2720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4652 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc27d0e9650, 0xc3c9823d40, 0xc3c9823da0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 1675 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0xc8, 0xc2aebb5700, 0xec, 0x100, 0xc2a6895800, 0xaa, 0x100, 0xc2778bdc80, 0xc2778bdbc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3335 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc39a663560, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc39a663560, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc39a663500, 0xc39a862000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc39a809910)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc399ec5308, 0xc39a862000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc399ec5308, 0xc39a67fa78, 0xc39a862000, 0x1000, 0x1000, 0x7ff9933f8010, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc39a85a0a0, 0xc39a862000, 0x1000, 0x1000, 0xc1eee13ace, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc39a787f20)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc39a787f20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc39a67fa20)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4919 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16ccb40, 0x64, 0xc3df5de300, 0xec, 0x100, 0xc3df357800, 0xc9, 0x100, 0xc3df648780, 0xc3df6486c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1049 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5ad80, 0x0, 0xc23fe67700, 0xea, 0x100, 0xc299878800, 0xc1, 0x100, 0xc299cd6f00, 0xc299cd6e40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4654 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16ccb40, 0x0, 0xc29c79ea00, 0xea, 0x100, 0xc3c9aed800, 0xb8, 0x100, 0xc3c9f82180, 0xc3c9f820c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 668 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6d80, 0x12d, 0xc226ca7f00, 0xec, 0x100, 0xc23c346800, 0xd0, 0x100, 0xc20c6cf1a0, 0xc20c6cf0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1352 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c67fd40, 0x0, 0xc226ca6f00, 0xea, 0x100, 0xc2950aa000, 0xb8, 0x100, 0xc230581c80, 0xc230581bc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3325 [select]:
net/http.(*persistConn).writeLoop(0xc39a67f810)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4350 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165b00, 0x64, 0xc39ed6f100, 0xeb, 0x100, 0xc2ed51d500, 0x1a, 0x20, 0xc373e12fc0, 0xc373e12f00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1210 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c67fd40, 0x64, 0xc2a55c7300, 0xec, 0x100, 0xc2a5a04400, 0x59, 0x80, 0xc2a5a59080, 0xc2a5a58fc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 370 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x12d, 0xc21a7b5200, 0xec, 0x100, 0xc22d3fe000, 0xcf, 0x100, 0xc2219c0a20, 0xc2219c0960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1660 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x64, 0xc261123a00, 0xec, 0x100, 0xc2acdda000, 0xc9, 0x100, 0xc2a0c5d8c0, 0xc2a0c5d800, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4478 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165d40, 0xca, 0xc2fd4b5600, 0xec, 0x100, 0xc3a0280000, 0xbc, 0x100, 0xc38e630ba0, 0xc38e630ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4762 [IO wait]:
net.(*pollDesc).Wait(0xc3cab7cb50, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3cab7cb50, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3cab7caf0, 0xc3ca96b000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3df2807b8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3caa641b0, 0xc3ca96b000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc3caa641b0, 0xc3cab7f708, 0xc3ca96b000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc287f1a8a0, 0xc3ca96b000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3cab9a5a0)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3cab9a5a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3cab7f6b0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4647 [select]:
net/http.(*persistConn).writeLoop(0xc3c50b3d90)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2720 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618d80, 0x64, 0xc36d6f7a00, 0xec, 0x100, 0xc372273800, 0xc9, 0x100, 0xc372917020, 0xc372916f60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2430 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605b00, 0x65, 0xc288777100, 0xec, 0x100, 0xc309b0f000, 0xc6, 0x100, 0xc35c459680, 0xc35c4595c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2288 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605b00, 0x64, 0xc32539f600, 0xea, 0x100, 0xc303594a80, 0xd, 0x10, 0xc34a4e57a0, 0xc34a4e56e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4928 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0dfae40, 0x3b9aca00, 0xc3a179d230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3a3cdee70, 0x7ff993413da0, 0xc3a3cd1380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7ff993413da0, 0xc3a3cd1380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 1650 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0x12c, 0xc20816fb00, 0xec, 0x100, 0xc2cae92800, 0xc4, 0x100, 0xc292b22660, 0xc292b225a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2121 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30bdefd40, 0x1, 0xc34c3a8d00, 0xea, 0x100, 0xc2a5acb000, 0xc7, 0x100, 0xc24dc928a0, 0xc24dc927e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2136 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30bdefd40, 0x66, 0xc2b9a5a700, 0xec, 0x100, 0xc347348000, 0xc8, 0x100, 0xc34c255c20, 0xc34c255b60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4956 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e12360)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 70 [select]:
net/http.(*persistConn).writeLoop(0xc208158370)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1402 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c7a2000, 0xc8, 0xc29c79f300, 0xec, 0x100, 0xc26c780800, 0xbb, 0x100, 0xc2d9e6afc0, 0xc2d9e6af00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 69 [runnable]:
net.(*pollDesc).Wait(0xc208011f70, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011f70, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208011f10, 0xc2081d5000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3d33ee260)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038220, 0xc2081d5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc208038220, 0xc2081583c8, 0xc2081d5000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc208367be0, 0xc2081d5000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e7f20)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e7f20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208158370)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 4683 [runnable]:
encoding/json.stateInString(0xc3e0dc5970, 0x69, 0x0)
	/usr/local/go/src/encoding/json/scanner.go:324
encoding/json.checkValid(0xc3cac4ee51, 0xa9, 0xa9, 0xc3e0dc5970, 0x0, 0x0)
	/usr/local/go/src/encoding/json/scanner.go:24 +0x123
encoding/json.Unmarshal(0xc3cac4ee51, 0xa9, 0xa9, 0x812be0, 0xc3e0e01800, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:72 +0x7c
github.com/couchbase/goxdcr/simple_utils.IsJSON(0xc3cac4ee51, 0xa9, 0xa9, 0xc3cfd1b3a8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:168 +0x6c
github.com/couchbase/goxdcr/parts.getDocMap(0xc3caca4ea0, 0xc3e0d19b00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:987 +0x4fe
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3a16ccd80, 0xa9012d, 0xc3cfd1bb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3a16ccd80, 0xc25752012d, 0xc3cfd1bb50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3a16ccd80, 0xc3c51408c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3a16ccd80, 0xc3c51408c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3a16ccd80, 0xc3a16d11f0, 0xc3a16ccf20, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 4958 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e124e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 129 [syscall, 4 minutes, locked to thread]:
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 1063 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5afc0, 0x12f, 0xc20816ed00, 0xec, 0x100, 0xc29b34e800, 0xce, 0x100, 0xc29b71b140, 0xc29b71b080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 719 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6d80, 0xcc, 0xc252ba4a00, 0xec, 0x100, 0xc25be1e800, 0xb7, 0x100, 0xc25d9cbb00, 0xc25d9cba40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4674 [runnable]:
github.com/couchbase/gomemcached/client.sendCommands(0xc27d3a24b0, 0xc3c545ba40, 0xc3c545baa0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:148 +0x283
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:173 +0x1df

goroutine 483 [IO wait]:
net.(*pollDesc).Wait(0xc22c0b15d0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc22c0b15d0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc22c0b1570, 0xc234e72000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3cec0e9b8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080f6120, 0xc234e72000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc2080f6120, 0xc212dcc268, 0xc234e72000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc233955900, 0xc234e72000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc233bbe360)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc233bbe360, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc212dcc210)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 268 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x2, 0xc211bb3a00, 0xea, 0x100, 0xc21337d000, 0xc0, 0x100, 0xc211377380, 0xc2113772c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 304 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x66, 0xc212500d00, 0xec, 0x100, 0xc2181bd000, 0xc8, 0x100, 0xc218b58c60, 0xc218b58ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2368 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605b00, 0x0, 0xc2d566f900, 0xea, 0x100, 0xc2fe8f8000, 0xb8, 0x100, 0xc3246877a0, 0xc3246876e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4935 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01a60, 0x1dcd6500, 0xc3a179d2f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2de190, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2de190)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1374 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c7a2000, 0x12e, 0xc2b1aaca00, 0xec, 0x100, 0xc2a6d6e800, 0x9c, 0x100, 0xc2db346960, 0xc2db3468a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3601 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38e955200, 0x12c, 0xc3aa6d2300, 0xec, 0x100, 0xc3a9cbf000, 0xc4, 0x100, 0xc3aa798c60, 0xc3aa798ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 733 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6b40, 0x66, 0xc226f92e00, 0xec, 0x100, 0xc2683c7800, 0xc8, 0x100, 0xc2687de120, 0xc2687de060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4616 [semacquire]:
sync.(*WaitGroup).Wait(0xc3a16ccce0)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc3a16ccb40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3a16ccb40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 1290 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c67fd40, 0x1, 0xc2b1aac400, 0xea, 0x100, 0xc2b0b85000, 0xc7, 0x100, 0xc2b1abb5c0, 0xc2b1abb500, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 756 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6d80, 0xcb, 0xc21a7b4500, 0xec, 0x100, 0xc26b29b800, 0xc9, 0x100, 0xc26b613800, 0xc26b613740, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4646 [IO wait]:
net.(*pollDesc).Wait(0xc3a5e5d6b0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3a5e5d6b0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3a5e5d650, 0xc3c5105000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3caefcd98)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc3a3cc2bc0, 0xc3c5105000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc3a3cc2bc0, 0xc3c50b3de8, 0xc3c5105000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2aa173da0, 0xc3c5105000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3c4f47740)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3c4f47740, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc3c50b3d90)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 975 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5afc0, 0xc8, 0xc288777c00, 0xec, 0x100, 0xc28d8f3000, 0xbb, 0x100, 0xc28df8afc0, 0xc28df8af00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 878 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5ad80, 0x0, 0xc276c56800, 0xea, 0x100, 0xc27629e800, 0x9c, 0x100, 0xc276cbb440, 0xc276cbb380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2740 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618fc0, 0x12d, 0xc373a65500, 0xec, 0x100, 0xc377e3c800, 0xcf, 0x100, 0xc378aea0c0, 0xc378aea000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 365 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x5, 0xc20816f300, 0xea, 0x100, 0xc218b8a000, 0xbc, 0x100, 0xc21e831f20, 0xc21e831e60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 484 [select]:
net/http.(*persistConn).writeLoop(0xc212dcc210)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 2076 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30c060000, 0x134, 0xc330c86900, 0xec, 0x100, 0xc33043b800, 0xc6, 0x100, 0xc330c932c0, 0xc330c93200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 709 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6b40, 0x65, 0xc252ba4500, 0xec, 0x100, 0xc252d95800, 0xc6, 0x100, 0xc2539c6c60, 0xc2539c6ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4199 [IO wait]:
net.(*pollDesc).Wait(0xc3ba56dbf0, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc3ba56dbf0, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc3ba56db90, 0xc2b2d8d000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc3d369c5a8)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc38e554d48, 0xc2b2d8d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc38e554d48, 0xc32aac4318, 0xc2b2d8d000, 0x1000, 0x1000, 0x8b53c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc37cb890e0, 0xc2b2d8d000, 0x1000, 0x1000, 0xc208014400, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc3801e4900)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc3801e4900, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc32aac42c0)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 397 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xcb, 0xc237b1a200, 0xec, 0x100, 0xc237012800, 0xc9, 0x100, 0xc237b586c0, 0xc237b58600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 393 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x64, 0xc210c25a00, 0xec, 0x100, 0xc2343f0800, 0xc9, 0x100, 0xc234e541e0, 0xc234e54120, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 340 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x68, 0xc211bb3100, 0xec, 0x100, 0xc2244ae000, 0xc2, 0x100, 0xc2245092c0, 0xc224509200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1322 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c7a2000, 0x12d, 0xc2cf2d3500, 0xec, 0x100, 0xc2d309c000, 0xcf, 0x100, 0xc2d31cb1a0, 0xc2d31cb0e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1735 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x68, 0xc3090b7600, 0xec, 0x100, 0xc30949e800, 0xc2, 0x100, 0xc30984a420, 0xc30984a360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1336 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c67fd40, 0x2, 0xc2ce921100, 0xea, 0x100, 0xc2ceb78000, 0xcf, 0x100, 0xc2cecbc360, 0xc2cecbc2a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2613 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618d80, 0x0, 0xc226f92800, 0xea, 0x100, 0xc36824f000, 0x93, 0x100, 0xc36873d020, 0xc36873cf60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1752 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0x12e, 0xc309b43b00, 0xec, 0x100, 0xc30a415000, 0xb3, 0x100, 0xc30a7aefc0, 0xc30a7aef00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1356 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c7a2000, 0xc9, 0xc234753e00, 0xec, 0x100, 0xc2781ac800, 0xbd, 0x100, 0xc2788d6b40, 0xc2788d6a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1398 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c67fd40, 0x65, 0xc2a67b1d00, 0xec, 0x100, 0xc29887b000, 0xc6, 0x100, 0xc2da0f2780, 0xc2da0f26c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 293 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xca, 0xc210c24b00, 0xec, 0x100, 0xc213f67800, 0x9e, 0x100, 0xc214369ec0, 0xc214369e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 344 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xc9, 0xc210c25300, 0xec, 0x100, 0xc224cf6800, 0xbd, 0x100, 0xc22550df80, 0xc22550dec0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 318 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0xc8, 0xc20802f100, 0xec, 0x100, 0xc21cef9000, 0xbb, 0x100, 0xc21dcd2ae0, 0xc21dcd2a20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 314 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d440, 0x0, 0xc210c25600, 0xea, 0x100, 0xc21bd65000, 0xb8, 0x100, 0xc21c4394a0, 0xc21c4393e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1368 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c67fd40, 0x64, 0xc2ce920900, 0xec, 0x100, 0xc2aebc0000, 0xab, 0x100, 0xc2dbb249c0, 0xc2dbb24900, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 306 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc20805d680, 0x12c, 0xc212501f00, 0xec, 0x100, 0xc2197a1800, 0xc4, 0x100, 0xc219f694a0, 0xc219f693e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4951 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcdf80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 1723 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x1, 0xc303087e00, 0xea, 0x100, 0xc304652800, 0xc7, 0x100, 0xc304e0b140, 0xc304e0b080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 679 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6b40, 0x1, 0xc234752700, 0xea, 0x100, 0xc23bbde000, 0xc7, 0x100, 0xc234701ec0, 0xc234701e00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 693 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6d80, 0xc8, 0xc252154a00, 0xec, 0x100, 0xc2518a7800, 0xa5, 0x100, 0xc252690120, 0xc252690060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4156 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2fc0, 0x12e, 0xc3b6101300, 0xec, 0x100, 0xc3a9340800, 0xb3, 0x100, 0xc3ba8eb3e0, 0xc3ba8eb320, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2066 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30bdefd40, 0x64, 0xc32539ef00, 0xec, 0x100, 0xc326c53800, 0xca, 0x100, 0xc327293320, 0xc327293260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4946 [runnable]:
sync.(*RWMutex).Lock(0xc3bc06e3f0)
	/usr/local/go/src/sync/rwmutex.go:87 +0xa4
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3bc06e3f0, 0x306c00b1, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:309 +0x56
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3a3cd1680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3a3cd1680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcdbc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4097 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2d80, 0x0, 0xc3badd3700, 0xea, 0x100, 0xc2a6898800, 0xa6, 0x100, 0xc34879fc20, 0xc34879fb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1026 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5afc0, 0x12c, 0xc288776700, 0xec, 0x100, 0xc28c963000, 0xa3, 0x100, 0xc2524b7320, 0xc2524b7260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2043 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30bdefd40, 0x65, 0xc32c172a00, 0xec, 0x100, 0xc332e34000, 0xc6, 0x100, 0xc332f28b40, 0xc332f28a80, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4562 [semacquire]:
sync.(*RWMutex).RLock(0xc208054c20)
	/usr/local/go/src/sync/rwmutex.go:36 +0x5f
github.com/couchbase/goxdcr/service_impl.(*ThroughSeqnoTrackerSvc).isPipelineRunning(0xc3a175ec00, 0x7ff99342c100)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/through_seqno_tracker_service.go:446 +0x64
github.com/couchbase/goxdcr/service_impl.(*ThroughSeqnoTrackerSvc).ProcessEvent(0xc3a175ec00, 0xc3e0c95d60, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/through_seqno_tracker_service.go:167 +0x4b
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).start(0xc25f2dfdb0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:74 +0x1cd
created by github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:57 +0x54

goroutine 3513 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38e955200, 0xc8, 0xc39ed6ff00, 0xec, 0x100, 0xc3a6611800, 0x55, 0x80, 0xc3a673e3c0, 0xc3a673e300, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2394 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605d40, 0xc9, 0xc29c79fb00, 0xec, 0x100, 0xc337ad8000, 0xbd, 0x100, 0xc2dcf5ade0, 0xc2dcf5ad20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 585 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6d80, 0xc8, 0xc237b1a300, 0xeb, 0x100, 0xc23c5f0c00, 0x2a, 0x40, 0xc23402c420, 0xc23402c360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1691 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0x132, 0xc2eaa57d00, 0xec, 0x100, 0xc2eaf24800, 0xbf, 0x100, 0xc2eba2a840, 0xc2eba2a780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1765 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x3, 0xc30b06ff00, 0xea, 0x100, 0xc30b1bf000, 0xb5, 0x100, 0xc30b5d72c0, 0xc30b5d7200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4148 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2fc0, 0xc8, 0xc335ebda00, 0xec, 0x100, 0xc37f4ee000, 0xbb, 0x100, 0xc380314f60, 0xc380314ea0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4940 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01ba0, 0x1dcd6500, 0xc3a179d470, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2c96d0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2c96d0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 1034 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5afc0, 0xc9, 0xc25e433200, 0xec, 0x100, 0xc299167800, 0xbd, 0x100, 0xc299473ce0, 0xc299473c20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 560 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6b40, 0x2, 0xc23fe67000, 0xea, 0x100, 0xc2122d8000, 0x3b, 0x40, 0xc2354e60c0, 0xc2354e6000, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1580 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x65, 0xc212501b00, 0xec, 0x100, 0xc2d4c7b000, 0xc6, 0x100, 0xc2a9f98120, 0xc2a9f98060, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1077 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5ad80, 0x6c, 0xc28898eb00, 0xec, 0x100, 0xc29bb03000, 0xb9, 0x100, 0xc29be211a0, 0xc29be210e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1223 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc29c7a2000, 0x12e, 0xc2a5061900, 0xeb, 0x100, 0xc2a62b6600, 0x26, 0x40, 0xc2a62af680, 0xc2a62af5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 721 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6b40, 0x64, 0xc252ba5600, 0xec, 0x100, 0xc26031f000, 0xc9, 0x100, 0xc260d586c0, 0xc260d58600, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 739 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2400e6d80, 0x12f, 0xc252155300, 0xec, 0x100, 0xc25c2af800, 0xce, 0x100, 0xc2140edc20, 0xc2140edb60, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4450 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165b00, 0x65, 0xc33ca91400, 0xec, 0x100, 0xc387a9c800, 0xc6, 0x100, 0xc2aa9a8840, 0xc2aa9a8780, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4937 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01ae0, 0x1dcd6500, 0xc3a179d4e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2c0be0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2c0be0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3534 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38e954fc0, 0x0, 0xc3a6765900, 0xea, 0x100, 0xc3a839f800, 0xa1, 0x100, 0xc3a85b3920, 0xc3a85b3860, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4932 [runnable]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e019a0, 0x1dcd6500, 0xc3a179d380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2d7a90, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2d7a90)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2400 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605b00, 0x64, 0xc2eaa57000, 0xec, 0x100, 0xc2f075a800, 0xc6, 0x100, 0xc333405500, 0xc333405440, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2748 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618fc0, 0xc8, 0xc3846ee000, 0xec, 0x100, 0xc383f6d800, 0xbb, 0x100, 0xc384687260, 0xc3846871a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1695 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x2, 0xc2ed9c5b00, 0xea, 0x100, 0xc2efd26800, 0xc4, 0x100, 0xc2f07ef980, 0xc2f07ef8c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4517 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165d40, 0x12c, 0xc2ce920200, 0xec, 0x100, 0xc3a26aa000, 0xc4, 0x100, 0xc3a2883320, 0xc3a2883260, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4942 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01c20, 0x1dcd6500, 0xc3a179d500, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2c1cc0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2c1cc0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2116 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30c060000, 0x130, 0xc351c24200, 0xec, 0x100, 0xc351819800, 0xbc, 0x100, 0xc351e64c60, 0xc351e64ba0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4955 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e122a0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3348 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc39a67fb80)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 1671 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca0138c0, 0x0, 0xc2da92aa00, 0xea, 0x100, 0xc2c1eb5000, 0xb8, 0x100, 0xc2b2474ba0, 0xc2b2474ae0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1559 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0xc8, 0xc252154900, 0xec, 0x100, 0xc2dbaf3800, 0x8a, 0x100, 0xc2ca956780, 0xc2ca9566c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 895 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5afc0, 0x12c, 0xc252ba5500, 0xec, 0x100, 0xc276589200, 0x32, 0x40, 0xc2766b39e0, 0xc2766b3920, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2142 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30c060000, 0xcb, 0xc33ca90400, 0xec, 0x100, 0xc34bfdf000, 0xc9, 0x100, 0xc352bf3440, 0xc352bf3380, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1028 [chan send, 4 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc26ce5ad80, 0x64, 0xc288777200, 0xec, 0x100, 0xc28491a000, 0xc9, 0x100, 0xc27b1cf7a0, 0xc27b1cf6e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4705 [runnable]:
syscall.Syscall(0x0, 0x2d, 0xc28bbbf880, 0x18, 0x18, 0x18, 0x0)
	/usr/local/go/src/syscall/asm_linux_amd64.s:21 +0x5
syscall.read(0x2d, 0xc28bbbf880, 0x18, 0x18, 0x72, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:867 +0x6e
syscall.Read(0x2d, 0xc28bbbf880, 0x18, 0x18, 0x1f0, 0x0, 0x0)
	/usr/local/go/src/syscall/syscall_unix.go:136 +0x58
net.(*netFD).Read(0xc3c9653880, 0xc28bbbf880, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/fd_unix.go:238 +0x314
net.(*conn).Read(0xc3a3cc2cd8, 0xc28bbbf880, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
io.ReadAtLeast(0x7ff99340bc98, 0xc3a3cc2cd8, 0xc28bbbf880, 0x18, 0x18, 0x18, 0x0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:298 +0xf1
io.ReadFull(0x7ff99340bc98, 0xc3a3cc2cd8, 0xc28bbbf880, 0x18, 0x18, 0x1f0, 0x0, 0x0)
	/usr/local/go/src/io/io.go:316 +0x6d
github.com/couchbase/gomemcached.(*MCRequest).Receive(0xc29c5cee78, 0x7ff99340bc98, 0xc3a3cc2cd8, 0xc28bbbf880, 0x18, 0x18, 0xc3e0d8b81f, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/mc_req.go:131 +0xee
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3bc06e3f0, 0xc3c9a9d560)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:402 +0x16a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 4119 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2d80, 0x64, 0xc37ec88200, 0xec, 0x100, 0xc382add800, 0xc9, 0x100, 0xc3837e8cc0, 0xc3837e8c00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 1730 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2ca013b00, 0x12d, 0xc308029b00, 0xec, 0x100, 0xc307e6a800, 0xcf, 0x100, 0xc3086d1e60, 0xc3086d1da0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4953 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e12120)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4933 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e019e0, 0x1dcd6500, 0xc3a179d460, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2c89b0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2c89b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 2417 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605d40, 0x12c, 0xc276c57d00, 0xec, 0x100, 0xc35d2b2000, 0xc4, 0x100, 0xc35d2d36e0, 0xc35d2d3620, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2732 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618d80, 0x0, 0xc37ec89f00, 0xea, 0x100, 0xc37e456800, 0xab, 0x100, 0xc37ee1aea0, 0xc37ee1ade0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2442 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605b00, 0x3, 0xc2ed9c4d00, 0xea, 0x100, 0xc35da4c800, 0xb5, 0x100, 0xc35e2ab680, 0xc35e2ab5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2408 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605d40, 0xc8, 0xc28898f500, 0xec, 0x100, 0xc2acf55000, 0xbc, 0x100, 0xc302e7bce0, 0xc302e7bc20, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2086 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30c060000, 0x136, 0xc33ca91500, 0xec, 0x100, 0xc33c6fd000, 0x1f4, 0x200, 0xc33d7032c0, 0xc33d703200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2374 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605d40, 0xca, 0xc2b1aad100, 0xec, 0x100, 0xc34c13c800, 0xbc, 0x100, 0xc2eee5d140, 0xc2eee5d080, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4677 [runnable]:
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc3bc0775f0, 0xc3c55120c0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:418 +0x36a
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeedWithConfig
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:332 +0x91

goroutine 3569 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38e955200, 0xc8, 0xc3a882f800, 0xec, 0x100, 0xc3a8a8f000, 0x9d, 0x100, 0xc3a8d4ca20, 0xc3a8d4c960, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4684 [semacquire]:
sync.(*WaitGroup).Wait(0xc3a16ccf20)
	/usr/local/go/src/sync/waitgroup.go:132 +0x169
github.com/couchbase/goxdcr/parts.(*CapiNozzle).onExit(0xc3a16ccd80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:604 +0x9a
github.com/couchbase/goxdcr/parts.*CapiNozzle.(github.com/couchbase/goxdcr/parts.onExit)·fm()
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:241 +0x27
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3a16ccd80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:110 +0x6b8
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4615 [runnable]:
encoding/json.(*decodeState).objectInterface(0xc3e0c470e0, 0xc3e0c492d0)
	/usr/local/go/src/encoding/json/decode.go:870 +0x3b
encoding/json.(*decodeState).object(0xc3e0c470e0, 0x833e80, 0xc3e0c492d0, 0xd4)
	/usr/local/go/src/encoding/json/decode.go:497 +0x3c8
encoding/json.(*decodeState).value(0xc3e0c470e0, 0x812be0, 0xc3e0c492d0, 0x16)
	/usr/local/go/src/encoding/json/decode.go:288 +0x3ce
encoding/json.(*decodeState).unmarshal(0xc3e0c470e0, 0x812be0, 0xc3e0c492d0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:149 +0x1e7
encoding/json.Unmarshal(0xc3cba19031, 0xab, 0xab, 0x812be0, 0xc3e0c492d0, 0x0, 0x0)
	/usr/local/go/src/encoding/json/decode.go:78 +0x145
github.com/couchbase/goxdcr/simple_utils.IsJSON(0xc3cba19031, 0xab, 0xab, 0xc3c9f013a8)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:168 +0x6c
github.com/couchbase/goxdcr/parts.getDocMap(0xc3cbb5aba0, 0xc3e0c4d800)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:987 +0x4fe
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs(0xc3a16ccb40, 0xa90064, 0xc3c9f01b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:783 +0x2fd
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocsWithRetry(0xc3a16ccb40, 0xc254c90064, 0xc3c9f01b50, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:742 +0xa9
github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchSendWithRetry(0xc3a16ccb40, 0xc3a5e6e8c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:568 +0x2fb
github.com/couchbase/goxdcr/parts.(*CapiNozzle).send_internal(0xc3a16ccb40, 0xc3a5e6e8c0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:460 +0x488
github.com/couchbase/goxdcr/parts.(*CapiNozzle).processData_batch(0xc3a16ccb40, 0xc3a16d10a0, 0xc3a16ccce0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:410 +0x9bd
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).Start
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:278 +0x42d

goroutine 2380 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605b00, 0x1, 0xc2a67b1800, 0xea, 0x100, 0xc343f2f000, 0xc7, 0x100, 0xc2e98fe7e0, 0xc2e98fe720, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4929 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0dfaec0, 0x3b9aca00, 0xc3a179d230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3a3cdee70, 0x7ff9934139f0, 0xc3a16ccb40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7ff9934139f0, 0xc3a16ccb40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 2090 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30bdefd40, 0x3, 0xc33dfb1800, 0xea, 0x100, 0xc33f5ca000, 0xb5, 0x100, 0xc33f5b17a0, 0xc33f5b16e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2064 [chan send, 3 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30c060000, 0xce, 0xc32539e300, 0xec, 0x100, 0xc32510d000, 0xab, 0x100, 0xc32545be60, 0xc32545bda0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2293 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc34d605d40, 0xc8, 0xc323457c00, 0xec, 0x100, 0xc331930000, 0xa3, 0x100, 0xc344a42420, 0xc344a42360, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4452 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165d40, 0x12e, 0xc32539e000, 0xec, 0x100, 0xc3861c9800, 0xb3, 0x100, 0xc2eac45740, 0xc2eac45680, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4430 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165b00, 0x64, 0xc351c25200, 0xec, 0x100, 0xc38ccf0000, 0xbe, 0x100, 0xc32339f680, 0xc32339f5c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4954 [chan receive]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e121e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 3347 [IO wait, 1 minutes]:
net.(*pollDesc).Wait(0xc39a82de90, 0x72, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc39a82de90, 0x0, 0x0)
	/usr/local/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc39a82de30, 0xc39a93e000, 0x1000, 0x1000, 0x0, 0x7ff993409df0, 0xc39a8bb098)
	/usr/local/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc39a79c0a0, 0xc39a93e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7ff99340bc98, 0xc39a79c0a0, 0xc39a67fbd8, 0xc39a93e000, 0x1000, 0x1000, 0x7ff9933f84c0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc39a893080, 0xc39a93e000, 0x1000, 0x1000, 0xc1eed863f4, 0x0, 0x0)
	<autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc39a93c300)
	/usr/local/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc39a93c300, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc39a67fb80)
	/usr/local/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:660 +0xc9f

goroutine 2132 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc30c060000, 0xcc, 0xc2b9a5a500, 0xec, 0x100, 0xc2eec6c800, 0xb0, 0x100, 0xc2f9342780, 0xc2f93426c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4515 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165b00, 0x1, 0xc2ce920000, 0xea, 0x100, 0xc3a7fb2800, 0xc7, 0x100, 0xc39fab7e00, 0xc39fab7d40, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4466 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165d40, 0x12d, 0xc309b42800, 0xec, 0x100, 0xc3782dd000, 0xcf, 0x100, 0xc2652532c0, 0xc265253200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2774 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618fc0, 0x12c, 0xc38c7b8200, 0xec, 0x100, 0xc38c7e4000, 0xc4, 0x100, 0xc38c851080, 0xc38c850fc0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 2760 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc35e618d80, 0x1, 0xc387fdce00, 0xea, 0x100, 0xc387de0800, 0xc7, 0x100, 0xc388290300, 0xc388290240, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4938 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01b20, 0x1dcd6500, 0xc3a179d360, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2dfdb0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2dfdb0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4462 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165b00, 0x0, 0xc30b06e000, 0xea, 0x100, 0xc37a104000, 0xb8, 0x100, 0xc2dc008360, 0xc2dc0082a0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4763 [select]:
net/http.(*persistConn).writeLoop(0xc3cab7f6b0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4930 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0dfaf40, 0x3b9aca00, 0xc3a179d230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3a3cdee70, 0x7ff993413da0, 0xc3a3cd1680, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7ff993413da0, 0xc3a3cd1680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 4931 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0dfafc0, 0x3b9aca00, 0xc3a179d230, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).stopPart(0xc3a3cdee70, 0x7ff9934139f0, 0xc3a16ccd80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x25c
github.com/couchbase/goxdcr/pipeline.func·004(0x7ff9934139f0, 0xc3a16ccd80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:363 +0x5a
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:368 +0x553

goroutine 4703 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3a3cd1680)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4936 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01aa0, 0x1dcd6500, 0xc3a179d4f0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f2c15e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f2c15e0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 3336 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc39a67fa20)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4934 [select]:
github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout(0xc3e0e01a20, 0x1dcd6500, 0xc3a179d2e0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:46 +0x534
github.com/couchbase/goxdcr/component.(*AsyncComponentEventListenerImpl).Stop(0xc25f503360, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x184
github.com/couchbase/goxdcr/pipeline.func·005(0x7ff993413e20, 0xc25f503360)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:374 +0x2e
created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Stop
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:375 +0x661

goroutine 4163 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2d80, 0x2, 0xc3abde4300, 0xea, 0x100, 0xc39cced800, 0xcf, 0x100, 0xc3b96c92c0, 0xc3b96c9200, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4950 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcdec0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4474 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc2d3165b00, 0x66, 0xc2fe4ed500, 0xec, 0x100, 0xc35e6d4800, 0xc8, 0x100, 0xc3ab883bc0, 0xc3ab883b00, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4675 [select]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc3a3cd1380)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75

goroutine 4200 [select]:
net/http.(*persistConn).writeLoop(0xc32aac42c0)
	/usr/local/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
	/usr/local/go/src/net/http/transport.go:661 +0xcbc

goroutine 4060 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3ac1c2fc0, 0x12c, 0xc3bb1bcc00, 0xec, 0x100, 0xc3b7a34800, 0xb0, 0x100, 0xc3bc283da0, 0xc3bc283ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 3580 [chan send, 1 minutes]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc38e954fc0, 0x64, 0xc3aa2b0400, 0xec, 0x100, 0xc3a933a800, 0xc9, 0x100, 0xc3aa2f5da0, 0xc3aa2f5ce0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4923 [runnable]:
time.appendUint(0xc35e727838, 0x5, 0x40, 0x4, 0x130, 0xa3c117, 0x16, 0x40)
	/usr/local/go/src/time/format.go:322
time.Time.Format(0xed0919231, 0x1b59ab7e, 0xeb0f40, 0xa3c117, 0x16, 0x0, 0x0)
	/usr/local/go/src/time/format.go:494 +0x13d3
github.com/couchbase/goxdcr/log.FormatTimeWithMilliSecondPrecision(0xed0919231, 0x1b59ab7e, 0xeb0f40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:226 +0x5f
github.com/couchbase/goxdcr/log.addTimestampToPrefix(0x9c66e0, 0x7, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:221 +0x64
github.com/couchbase/goxdcr/log.(*CommonLogger).logMsgf(0xc20802b5c0, 0x1, 0x9c66e0, 0x7, 0xaa5e10, 0x3c, 0xc35e727c60, 0x3, 0x3)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:137 +0x77
github.com/couchbase/goxdcr/log.(*CommonLogger).Infof(0xc20802b5c0, 0xaa5e10, 0x3c, 0xc35e727c60, 0x3, 0x3)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/log/logger.go:148 +0x76
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).processReport(0xc20814e1c0, 0xc3dfd901b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:325 +0x8aa
github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).waitForResponse(0xc20814e1c0, 0xc3dfd901b0, 0xc3dfd7ffb0, 0xc2080104d0, 0xc3dfd23ea0, 0xc2080e9ac0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:310 +0x5ae
created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).sendHeartBeats
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:233 +0x85f

goroutine 4832 [select]:
net/http.(*persistConn).roundTrip(0xc39a67f810, 0xc3e0c3ad00, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xe94600, 0xc3e0b88750, 0xc3e0d16040, 0x0, 0x0)
	/usr/local/go/src/net/http/transport.go:235 +0x558
net/http.send(0xc3e0b88750, 0x7ff993409fe8, 0xe94600, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xeb0d00, 0xc3e0b88750, 0x1b, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xeb0d00, 0xc3e0b88750, 0xb05678, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:367 +0xb25
net/http.(*Client).Do(0xeb0d00, 0xc3e0b88750, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/client.go:174 +0xa4
github.com/couchbase/go-couchbase.doHTTPRequest(0xc3e0b88750, 0x7ff99340dae0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:402 +0x76
github.com/couchbase/go-couchbase.queryRestAPI(0xc3e0c363f0, 0x9bbca0, 0x6, 0x7ff99340dae0, 0xc3e0d16000, 0x80f820, 0xc3e0bcc858, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:492 +0x286
github.com/couchbase/go-couchbase.(*Client).parseURLResponse(0xc3e0bcc840, 0x9bbca0, 0x6, 0x80f820, 0xc3e0bcc858, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:512 +0x75
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc3e0badfe0, 0x15, 0x7ff99340dae0, 0xc3e0d16000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/go-couchbase/pools.go:613 +0x132
github.com/couchbase/goxdcr/utils.LocalPool(0xc3e0c3aca0, 0xe, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:85 +0x263
github.com/couchbase/goxdcr/utils.LocalBucket(0xc3e0c3aca0, 0xe, 0xc20802ac78, 0x8, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/utils/utils.go:105 +0x166
github.com/couchbase/goxdcr/service_impl.(*ClusterInfoSvc).GetBucket(0xc2080380f0, 0x7ff99340de90, 0xc20801f440, 0xc20802ac78, 0x8, 0x40adf9, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/cluster_info_service.go:133 +0x25e
github.com/couchbase/goxdcr/factory.(*XDCRFactory).constructSourceNozzles(0xc208032280, 0xc3e0707100, 0xc20800b540, 0x32, 0xc3e0c3ac90, 0x1, 0xc259eebaa0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:280 +0x11c
github.com/couchbase/goxdcr/factory.(*XDCRFactory).NewPipeline(0xc208032280, 0xc20800b540, 0x32, 0xc3e0c3aba0, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:94 +0x39b
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).startPipeline(0xeb0fa0, 0xc20800b540, 0x32, 0x0, 0x0, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:280 +0x413
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).update(0xc3bc013800, 0xc3ccdd7c80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:593 +0xdc2
github.com/couchbase/goxdcr/pipeline_manager.(*pipelineUpdater).start(0xc3bc013800)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:569 +0x44a
created by github.com/couchbase/goxdcr/pipeline_manager.(*pipelineManager).launchUpdater
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_manager/pipeline_manager.go:443 +0x41c

goroutine 4901 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208067080, 0x7ff99340d910, 0xc3e0b60000, 0xc3dfd625b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080380c0, 0x7ff99340d910, 0xc3e0b60000, 0xc3dfd625b0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7ff99340d910, 0xc3e0b60000, 0xc3dfd625b0)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080670e0, 0x7ff99340d910, 0xc3e0b60000, 0xc3dfd625b0)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3d91f14a0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 4911 [runnable]:
net/url.getscheme(0xc3e056d3b4, 0x1f, 0xc3e056d3b0, 0xc3e056d3b0, 0xc208321000, 0x7ff9736e0080, 0x1000, 0x640f62)
	/usr/local/go/src/net/url/url.go:294
net/url.parse(0xc3e056d3b4, 0x1f, 0x1, 0xc3e0c9a850, 0x0, 0x0)
	/usr/local/go/src/net/url/url.go:378 +0x350
net/url.ParseRequestURI(0xc3e056d3b4, 0x1f, 0x1, 0x0, 0x0)
	/usr/local/go/src/net/url/url.go:355 +0x49
net/http.ReadRequest(0xc352ddbaa0, 0xc3dfd624e0, 0x0, 0x0)
	/usr/local/go/src/net/http/request.go:632 +0x6c3
net/http.(*conn).readRequest(0xc3df14cbe0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc3df14cbe0)
	/usr/local/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 4913 [chan receive]:
github.com/couchbase/goxdcr/adminport.(*httpServer).systemHandler(0xc208067080, 0x7ff99340d910, 0xc3df14ce60, 0xc3e0127c70)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:136 +0x1cf
github.com/couchbase/goxdcr/adminport.(*Handler).ServeHTTP(0xc2080380c0, 0x7ff99340d910, 0xc3df14ce60, 0xc3e0127c70)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/adminport/admin_httpd.go:184 +0x1be
net/http.(*ServeMux).ServeHTTP(0xc20803c8d0, 0x7ff99340d910, 0xc3df14ce60, 0xc3e0127c70)
	/usr/local/go/src/net/http/server.go:1541 +0x17d
net/http.serverHandler.ServeHTTP(0xc2080670e0, 0x7ff99340d910, 0xc3df14ce60, 0xc3e0127c70)
	/usr/local/go/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc3df14cdc0)
	/usr/local/go/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:1751 +0x35e

goroutine 4906 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16ccd80, 0x12d, 0xc3ddb1a800, 0xec, 0x100, 0xc3de961800, 0xcf, 0x100, 0xc3dea151a0, 0xc3dea150e0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4866 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16ccb40, 0x0, 0xc2383dca00, 0xea, 0x100, 0xc3ce4d6400, 0x5d, 0x80, 0xc3ceb34540, 0xc3ceb34480, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4944 [chan send]:
github.com/couchbase/gomemcached/client.(*UprFeed).CloseStream(0xc3bc0775f0, 0x306b0003, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/godeps/src/github.com/couchbase/gomemcached/client/upr_feed.go:320 +0x268
github.com/couchbase/goxdcr/parts.(*DcpNozzle).closeUprStreams(0xc3a3cd1380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:309 +0x327
github.com/couchbase/goxdcr/parts.(*DcpNozzle).Stop(0xc3a3cd1380, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:280 +0x209
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcda40)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4947 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3a16ccd80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3a16ccd80, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcdc80)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4880 [chan send]:
github.com/couchbase/goxdcr/parts.(*CapiNozzle).writeDocs(0xc3a16ccb40, 0x2, 0xc3d5048f00, 0xea, 0x100, 0xc3d46f5800, 0xcf, 0x100, 0xc3d55c1380, 0xc3d55c12c0, ...)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:883 +0x6b6
created by github.com/couchbase/goxdcr/parts.(*CapiNozzle).batchUpdateDocs
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:833 +0xd31

goroutine 4952 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0e12060)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4949 [runnable]:
github.com/couchbase/goxdcr/component.*AsyncComponentEventListenerImpl.(github.com/couchbase/goxdcr/component.stop)·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/component/async_listener.go:85 +0x97
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcde00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182

goroutine 4945 [chan receive]:
github.com/couchbase/goxdcr/gen_server.(*GenServer).Stop_server(0xc3a16ccb40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:147 +0x223
github.com/couchbase/goxdcr/parts.(*CapiNozzle).Stop(0xc3a16ccb40, 0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/capi_nozzle.go:313 +0x56c
github.com/couchbase/goxdcr/common.Part.Stop·fm(0x0, 0x0)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:288 +0x4b
github.com/couchbase/goxdcr/simple_utils.func·001(0xc3e0bcdb00)
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:38 +0x59
created by github.com/couchbase/goxdcr/simple_utils.ExecWithTimeout
	/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/simple_utils/simple_utils.go:40 +0x182
[goport] 2017/04/25 13:53:39 /opt/couchbase/bin/goxdcr terminated: exit status 2
ReplicationManager 2017-04-25T13:53:40.081-05:00 [INFO] GOMAXPROCS=2
AuditService 2017-04-25T13:53:40.081-05:00 [INFO] Created audit service.
UILogService 2017-04-25T13:53:40.081-05:00 [INFO] Created ui log service.
ReplicationSpecService 2017-04-25T13:53:40.108-05:00 [INFO] Init cache for ReplicationSpecService...
ReplicationSpecService 2017-04-25T13:53:40.113-05:00 [INFO] Cache is initialized for ReplicationSpecService
GenericSupervisor 2017-04-25T13:53:40.113-05:00 [INFO] Adding child PipelineMasterSupervisor to supervisor ReplicationManagerSupervisor
PipelineManager 2017-04-25T13:53:40.113-05:00 [INFO] Pipeline Manager is constucted
ReplicationManager 2017-04-25T13:53:40.113-05:00 [INFO] Replication manager is initialized
GenericSupervisor 2017-04-25T13:53:40.113-05:00 [INFO] Starting supervisor PipelineMasterSupervisor.
GenericSupervisor 2017-04-25T13:53:40.113-05:00 [INFO] Started supervisor PipelineMasterSupervisor.
ReplicationManager 2017-04-25T13:53:40.113-05:00 [INFO] Master supervisor has started
GenericSupervisor 2017-04-25T13:53:40.113-05:00 [INFO] Starting supervisor ReplicationManagerSupervisor.
GenericSupervisor 2017-04-25T13:53:40.113-05:00 [INFO] Started supervisor ReplicationManagerSupervisor.
ReplicationManager 2017-04-25T13:53:40.113-05:00 [INFO] pollEOF: About to start stdin polling
ReplicationSpecChangeListener 2017-04-25T13:53:40.113-05:00 [INFO] Started MetakvChangeListener ReplicationSpecChangeListener
RemoteClusterChangeListener 2017-04-25T13:53:40.113-05:00 [INFO] Started MetakvChangeListener RemoteClusterChangeListener
ReplicationManager 2017-04-25T13:53:40.113-05:00 [INFO] initPausedReplications succeeded
AdminPort 2017-04-25T13:53:40.113-05:00 [INFO] Constructed adminport
ReplicationManager 2017-04-25T13:53:40.113-05:00 [INFO] Admin port has been launched
GenericSupervisor 2017-04-25T13:53:40.113-05:00 [INFO] Adding child AdminportSupervisor to supervisor ReplicationManagerSupervisor
ReplicationManager 2017-04-25T13:53:40.113-05:00 [INFO] ReplicationManager is running
HttpServer 2017-04-25T13:53:40.114-05:00 [INFO] [xdcr:127.0.0.1:9998] new http server xdcr 127.0.0.1:9998 /
AdminPort 2017-04-25T13:53:40.114-05:00 [INFO] http server started 127.0.0.1:9998 !
HttpServer 2017-04-25T13:53:40.114-05:00 [INFO] [xdcr:127.0.0.1:9998] starting ...
RemoteClusterChangeListener 2017-04-25T13:53:40.118-05:00 [INFO] metakvCallback called on listener RemoteClusterChangeListener with path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
RemoteClusterService 2017-04-25T13:53:40.118-05:00 [INFO] metakvCallback called on path = /remoteCluster/UTGoIhQ3Fqrk6DsYP8HWCMYuztQTvnhLS6ZiiWT-AN0=
ReplicationSpecChangeListener 2017-04-25T13:53:40.119-05:00 [INFO] metakvCallback called on listener ReplicationSpecChangeListener with path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
ReplicationSpecService 2017-04-25T13:53:40.119-05:00 [INFO] ReplicationSpecServiceCallback called on path = /replicationSpec/f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:53:40.402-05:00 [INFO] rep_status for topic f4268e62702130298bf87f17cc481219/metadata/metadata is nil. Initialize it
PipelineManager 2017-04-25T13:53:55.254-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is broken, but not yet attended, launch updater
PipelineManager 2017-04-25T13:53:55.260-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
PipelineManager 2017-04-25T13:53:55.260-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[]}, progress={}
]
PipelineManager 2017-04-25T13:53:55.260-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
PipelineManager 2017-04-25T13:53:55.261-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=<nil> 
PipelineManager 2017-04-25T13:53:55.261-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:53:55.261-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:53:55.261-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is not running
PipelineManager 2017-04-25T13:53:55.261-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
XDCRFactory 2017-04-25T13:53:55.363-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:53:55.363-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:53:55.363-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:53:55.412-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:53:55.412-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:53:55.413-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:53:55.413-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:53:55.413-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:53:55.413-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:53:55.413-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:53:55.416-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:53:55.421-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
GenericSupervisor 2017-04-25T13:53:55.423-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:53:55.423-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
GenericSupervisor 2017-04-25T13:53:55.423-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:55.423-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:55.423-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:55.423-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:53:55.423-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:53:55.423-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
CheckpointManager 2017-04-25T13:53:55.423-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
CapiUtils 2017-04-25T13:53:55.452-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:53:55.452-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:53:55.452-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:53:55.453-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
StatisticsManager 2017-04-25T13:53:55.453-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:53:55.453-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
XDCRFactory 2017-04-25T13:53:55.453-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:53:55.454-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545 is constructed, start it
GenericPipeline 2017-04-25T13:53:55.458-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[227 238 265 289 213 337 241 291 334 339 210 219 304 176 200 316 333 309 275 193 320 216 243 253 328 214 234 301 192 271 311 290 184 248 276 279 335 287 326 245 247 295 308 173 182 228 286 261 312 322 218 189 204 209 272 277 302 266 274 296 226 324 221 252 281 306 202 299 313 231 232 174 317 284 171 278 336 314 323 330 203 205 239 196 251 340 263 282 190 257 327 181 175 297 236 260 268 270 199 235 319 180 206 259 254 223 246 207 237 294 269 283 217 280 255 201 332 183 250 177 307 305 310 212 215 187 197 244 321 178 233 264 211 229 293 331 194 256 273 325 172 292 300 208 303 188 285 288 315 249 298 186 258 318 220 222 230 329 185 240 198 267 195 242 179 224 262 338 191 225] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[18 64 110 152 47 94 80 120 11 32 129 40 103 115 136 154 67 116 131 125 132 1 49 127 98 117 24 33 39 74 84 113 90 91 133 167 10 28 45 76 145 6 52 73 96 147 155 27 78 130 138 50 59 142 159 170 0 35 85 102 25 134 23 30 157 86 106 109 123 75 124 140 29 31 43 153 63 82 87 146 5 26 89 166 161 54 2 58 61 137 13 42 51 163 38 19 20 21 135 83 128 44 108 139 69 71 97 119 3 15 126 34 37 114 141 7 151 9 57 66 164 81 8 46 60 68 41 111 56 79 101 4 22 107 62 165 70 93 105 162 122 148 160 55 95 150 169 16 36 12 17 53 100 149 121 72 112 48 99 143 158 92 144 65 156 77 88 104 14 118 168]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[128 83 108 139 44 71 97 119 69 3 126 15 37 114 141 34 151 7 57 66 164 9 46 60 68 81 8 111 41 79 101 56 22 107 4 62 93 105 162 165 70 122 160 148 95 55 169 150 36 16 17 53 100 12 149 121 112 72 99 143 158 48 144 92 156 65 77 104 88 14 168 118 64 110 18 152 94 47 120 80 32 129 11 103 115 40 154 136 116 131 67 132 125 49 127 1 33 39 74 98 117 24 113 84 28 45 76 90 91 133 167 10 52 73 96 145 6 78 130 138 147 155 27 59 142 50 35 85 102 159 170 0 134 25 30 157 23 106 109 123 86 124 140 75 31 43 153 29 82 87 63 146 26 89 166 5 161 54 58 61 137 2 42 51 163 13 38 20 21 135 19] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[190 257 263 282 181 327 175 297 199 235 236 260 268 270 319 180 206 259 223 246 254 207 237 294 269 283 217 280 255 183 201 332 250 177 307 305 310 187 197 212 215 178 233 244 321 211 229 264 293 331 194 256 273 325 172 292 300 208 188 285 303 288 315 249 186 258 298 220 222 230 318 185 240 329 267 198 195 242 224 262 338 179 191 225 213 227 238 265 289 337 241 291 334 210 219 339 176 200 304 309 316 333 193 275 216 243 320 253 214 234 328 301 192 271 311 184 248 290 276 279 335 287 326 173 245 247 295 308 182 228 286 218 261 312 322 189 204 209 272 277 302 266 274 296 221 226 324 252 281 306 202 299 313 231 232 174 317 284 171 278 336 203 205 239 314 323 330 196 251 340]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: doc_batch_size_kb:2048 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 active:true checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:53:55.458-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545...
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:53:55.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
ToplogyChangeDetector 2017-04-25T13:53:55.468-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:53:55.468-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:53:55.468-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:53:55.468-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:53:55.468-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:53:55.468-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:53:55.468-05:00 [INFO] Checkpointing starts in 1803.081 sec
PipelineRuntimeCtx 2017-04-25T13:53:55.468-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:53:55.468-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:53:55.468-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:53:55.468-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:53:55.473-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:53:55.473-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:53:55.473-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:53:55.499-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:53:55.580-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:53:55.580-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:53:55.580-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:53:55.658-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
TCPConnPoolMgr 2017-04-25T13:53:55.664-05:00 [INFO] Create TCP Pool - poolName=Couch_Capi_10.202.98.33:9091,
TCPConnPoolMgr 2017-04-25T13:53:55.664-05:00 [INFO] connectionSize=5
TCPConnPoolMgr 2017-04-25T13:53:55.667-05:00 [INFO] Connection pool Couch_Capi_10.202.98.33:9091 is created with 5 clients
CapiNozzle 2017-04-25T13:53:55.667-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:53:55.667-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:53:55.667-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:53:55.676-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:53:55.676-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:53:55.676-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:53:55.758-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:53:55.759-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:53:55.759-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:53:55.759-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:53:55.759-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:53:55.759-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:53:55.759-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:53:55.861-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:53:55.861-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:53:55.861-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:53:55.880-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:53:55.880-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:53:56.033-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:53:56.033-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:53:56.033-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:53:56.033-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:53:56.033-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:53:56.033-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:53:56.033-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545 is started----------
PipelineManager 2017-04-25T13:53:56.034-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:53:56.034-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:53:56.034-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:53:56.578-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:53:57.030-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:53:57.031-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:53:57.031-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:53:57.031-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:53:57 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 18 successful
StatisticsManager 2017-04-25T13:53:57.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:53:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:53:57 UPR_STREAMREQ for vb 246 successful
CapiNozzle 2017-04-25T13:53:58.065-05:00 [INFO] Send batch count=187
2017/04/25 13:53:58 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 260 successful
CapiNozzle 2017-04-25T13:53:58.146-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:53:58.153-05:00 [INFO] Send batch count=174
2017/04/25 13:53:58 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 196 successful
CheckpointManager 2017-04-25T13:53:58.348-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
CheckpointManager 2017-04-25T13:53:58.348-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:53:58 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:53:58.399-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:53:58.423-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:53:58 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:53:58 UPR_STREAMREQ for vb 198 successful
DcpNozzle 2017-04-25T13:53:58.459-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:53:58.539-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:53:58.540-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
DcpNozzle 2017-04-25T13:53:58.548-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:53:58.553-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:53:58.714-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:53:58.729-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:53:58.865-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:53:59.027-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:53:59.066-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:53:59.078-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:53:59.325-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:53:59.344-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:53:59.434-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:53:59.434-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:53:59.434-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:53:59.434-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:53:59.434-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:53:59.444-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:53:59.444-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:53:59.448-05:00 [INFO] err_list=[{"time":"2017-04-25T13:53:59.448181132-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:53:59.448-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:53:59.473-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:53:59.474-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545 {"Errors": "[{\"time\":\"2017-04-25T13:53:59.448181132-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 13, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 142, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 6477, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 142, "set_filtered": 0, "set_received_from_dcp": 6477, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:53:59.501-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:59.501-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:59.512-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:53:59.614-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:53:59.614-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:53:59.615-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:53:59.615-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:53:59.615-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:53:59.716-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:53:59.716-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:53:59.727-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:53:59.758-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:53:59.788-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:54:00.048-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:54:00.067-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:54:00.269-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:00.286-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:54:00.551-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:54:00.562-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:54:00.633-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:00.633-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:54:00.633-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:54:00.633-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:00.633-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:00.718-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:00.718-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:00.736-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:54:00.879-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:54:00.879-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:54:00.879-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:54:00.879-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:00.879-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:00.986-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:00.987-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:00.999-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:54:01.200-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:01.213-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:54:01.455-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:01.482-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:54:01.922-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:01.940-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:54:02.175-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:02.189-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:54:02.454-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:02.454-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:54:02.454-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:54:02.454-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:02.454-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:02.540-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:02.540-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:02.592-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:54:02.662-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:02.662-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:54:02.662-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:54:02.662-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:02.662-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:02.735-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:02.735-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:02.756-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:54:03.244-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:03.265-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:54:03.467-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:54:03.489-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:54:03.648-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:03.663-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:54:03.765-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:54:03.787-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:54:03.995-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:03.995-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:54:03.995-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:54:03.995-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:03.995-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:04.118-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:04.119-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:04.137-05:00 [INFO] Retrying update docs for vb 106 for the 1th time
CapiNozzle 2017-04-25T13:54:04.253-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:54:04.254-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:54:04.254-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:54:04.254-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:04.254-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:04.387-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:04.387-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:04.406-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:54:04.693-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:54:04.711-05:00 [INFO] Retrying update docs for vb 106 for the 2th time
CapiNozzle 2017-04-25T13:54:05.067-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:05.096-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:54:05.325-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:54:05.338-05:00 [INFO] Retrying update docs for vb 106 for the 3th time
GenericSupervisor 2017-04-25T13:54:05.474-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:54:05.474-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:54:05.474-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:54:05.779-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:54:05.933-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:05.944-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:54:06.255-05:00 [ERROR] batchUpdateDocs for vb 106 failed with err receieved unexpected status code, 500, from update docs request for vb 106
.
CapiNozzle 2017-04-25T13:54:06.255-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 106 after 3 retries
CapiNozzle 2017-04-25T13:54:06.255-05:00 [ERROR] Raise error condition batch update docs failed for vb 106 after 3 retries
GenericSupervisor 2017-04-25T13:54:06.255-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:06.255-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:06.328-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:06.328-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:06.345-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:54:06.732-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:06.732-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:54:06.732-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:54:06.732-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:06.732-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:06.835-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:06.835-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:06.849-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:54:06.855-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:06.868-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:54:07.166-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:07.186-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:54:07.324-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:07.347-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:54:07.625-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:07.649-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:54:07.821-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:07.821-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:54:07.821-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:54:07.821-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:07.821-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:07.915-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:07.915-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:07.933-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:54:08.080-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:08.080-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:54:08.080-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:54:08.080-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:08.080-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:08.149-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:08.149-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:08.180-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:54:08.267-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:08.288-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:54:08.522-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:54:08.534-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:54:08.749-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:08.759-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:54:09.068-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:54:09.079-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:54:09.230-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:09.230-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:54:09.230-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:54:09.230-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:09.230-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:09.340-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:09.340-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:09.351-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
PipelineManager 2017-04-25T13:54:09.448-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:54:09.448-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:54:09.448-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:09.448-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
PipelineRuntimeCtx 2017-04-25T13:54:09.448-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:54:09.448-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:09.448-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:54:09.448-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:09.448-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:54:09.448-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:54:09.448-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:54:09.448-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:54:09.449-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:09.449-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:09.449068735 -0500 CDT
GenericSupervisor 2017-04-25T13:54:09.449-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:54:09.449-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:09.449-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:54:09.449-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-413376545 has been stopped
GenericPipeline 2017-04-25T13:54:09.449-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-413376545 is stopped
PipelineManager 2017-04-25T13:54:09.449-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:54:09.449-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:53:59.448181132-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:54:09.449-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:09.449-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-413376545
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:54:09.450-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:09.450-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:09.450-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:09.450-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:54:09.450-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:54:09.450-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
GenericPipeline 2017-04-25T13:54:09.450-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:09.450-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.450-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:09.451-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:09.451-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:09.450769194 -0500 CDT
DcpNozzle 2017-04-25T13:54:09.451-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:09.451-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:54:09.451-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:09.451044479 -0500 CDT
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.451-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:09.451-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:54:09.452-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:54:09.452-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:54:09.452-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:54:09.452-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:54:09.452-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:54:09.452-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:09.452-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:54:09.452-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:09.452-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:54:09.452-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:09.452-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:54:09.453-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:09.453-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:09.453351804 -0500 CDT
DcpNozzle 2017-04-25T13:54:09.453-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:54:09.453-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:09.453-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:54:09.455-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:09.455-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:09.455253194 -0500 CDT
DcpNozzle 2017-04-25T13:54:09.455-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:54:09.455-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:09.455-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:54:09 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 173 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 174 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 175 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 176 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:54:09 Stream Closed for vb 3 StreamEnd simulated
XDCRFactory 2017-04-25T13:54:09.574-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:09.574-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:54:09.574-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:54:09.650-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:09.650-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:54:09.650-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:09.650-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:54:09.651-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:09.651-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:54:09.651-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:54:09.651-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:54:09.652-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
GenericSupervisor 2017-04-25T13:54:09.654-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:54:09.654-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
GenericSupervisor 2017-04-25T13:54:09.654-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:09.654-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:09.654-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:09.654-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:09.654-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:54:09.654-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
CheckpointManager 2017-04-25T13:54:09.654-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
CapiUtils 2017-04-25T13:54:09.684-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:09.684-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:09.684-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:54:09.685-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
StatisticsManager 2017-04-25T13:54:09.686-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:54:09.686-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
XDCRFactory 2017-04-25T13:54:09.686-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:54:09.686-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223 is constructed, start it
GenericPipeline 2017-04-25T13:54:09.688-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[112 153 40 106 11 26 58 91 0 138 12 4 37 102 137 55 141 70 81 121 35 142 158 150 167 2 21 52 145 13 53 61 49 86 10 72 96 131 60 73 22 78 111 16 20 28 41 103 120 59 95 133 62 51 97 25 43 136 160 89 157 3 7 8 17 98 115 71 80 109 149 66 114 161 129 46 104 110 119 170 64 143 14 27 154 166 6 24 32 125 148 155 163 1 50 84 92 134 77 105 168 117 128 140 113 48 169 75 135 156 159 9 69 90 34 83 88 164 85 127 39 47 99 57 124 165 56 107 144 30 74 94 130 147 44 126 42 132 63 93 146 5 108 118 38 54 67 116 45 76 82 87 123 162 15 18 33 36 100 151 65 101 31 23 122 19 79 29 68 139 152] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[202 225 296 305 279 311 318 176 191 171 181 197 288 233 239 276 315 336 179 203 258 316 183 198 255 269 273 327 322 241 303 310 178 194 274 172 217 229 250 264 335 211 261 252 307 201 195 221 226 331 243 260 302 325 209 189 213 242 292 222 333 204 208 238 278 337 216 248 275 234 257 334 254 256 294 267 326 308 193 228 299 340 207 190 246 259 265 220 174 205 280 284 182 212 215 300 268 272 290 218 232 173 184 185 188 328 196 287 291 330 298 199 206 245 271 286 321 323 282 317 319 314 214 227 285 313 332 320 187 240 244 251 324 247 262 192 224 263 338 289 249 219 231 295 236 175 230 235 283 266 270 329 177 281 293 297 186 237 210 223 301 304 339 180 312 200 277 306 309 253]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[248 275 257 334 234 254 256 294 267 326 193 228 299 308 207 340 265 190 246 259 174 205 220 280 284 212 215 300 182 268 272 290 218 184 185 188 232 173 328 330 196 287 291 245 271 286 298 199 206 321 323 317 319 282 227 285 313 314 214 332 240 244 251 320 187 324 247 262 263 338 192 224 289 249 231 295 219 175 230 235 236 283 266 270 329 177 281 293 297 186 237 304 339 210 223 301 180 312 306 200 277 309 253 296 305 202 225 279 311 318 176 191 288 171 181 197 233 239 315 336 276 203 258 316 179 183 198 327 255 269 273 241 303 310 322 194 274 178 172 217 229 250 264 335 211 261 252 307 195 201 221 226 331 243 260 302 325 209 189 213 242 292 222 333 238 278 204 208 216 337] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[71 80 109 149 66 114 161 129 170 46 104 110 119 64 143 14 27 154 166 148 155 6 24 32 125 163 134 1 50 84 92 77 105 168 117 128 140 113 48 169 75 135 156 159 9 69 90 34 83 88 164 85 127 39 47 99 57 124 165 56 107 144 147 30 74 94 130 44 126 42 132 63 93 146 5 108 118 38 54 67 116 45 76 82 87 123 162 100 15 18 33 36 151 65 101 31 23 122 19 79 29 68 139 152 112 153 40 106 11 26 58 91 0 138 12 4 37 102 137 55 141 70 81 121 35 142 158 150 167 2 21 52 145 13 53 61 49 86 10 72 96 131 60 73 22 78 111 103 120 16 20 28 41 59 95 133 62 51 97 25 43 136 160 89 157 98 115 3 7 8 17]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[log_level:Info stats_interval:1000 filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 replication_type:capi doc_batch_size_kb:2048 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:54:09.688-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223...
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:09.699-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
CheckpointManager 2017-04-25T13:54:09.699-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:54:09.699-05:00 [INFO] Checkpointing starts in 1804.608 sec
PipelineRuntimeCtx 2017-04-25T13:54:09.699-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:54:09.699-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:54:09.699-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:54:09.702-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:54:09.702-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:54:09.702-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:54:09.703-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
StatisticsManager 2017-04-25T13:54:09.702-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:54:09.703-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:54:09.703-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:54:09.703-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:54:09.703-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:54:09.729-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:54:09.730-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:54:09.730-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:54:09.732-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:09.732-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:09.732-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:54:09.742-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:54:09.742-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:54:09.744-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:54:09.749-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:54:09.796-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:54:09.809-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:54:09.809-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:09.809-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:54:09.810-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:54:09.810-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:54:09.810-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:54:09.875-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:54:10.313-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:54:10.313-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:54:10.313-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:54:10.345-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:54:10.345-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
PipelineManager 2017-04-25T13:54:10.486-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:53:59.448181132-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={The runtime context is started}
]
DcpNozzle 2017-04-25T13:54:10.514-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:54:10.515-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:10.515-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:54:10.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:54:10.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:54:10.515-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:54:10.515-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223 is started----------
PipelineManager 2017-04-25T13:54:10.515-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:54:10.515-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:54:10.515-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:54:10.642-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:54:10.643-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:54:10.643-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:54:10.643-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
StatisticsManager 2017-04-25T13:54:10.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:54:10 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:54:10.808-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:54:10.808-05:00 [ERROR] Received malformed response from tcp connection vb=1. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f104/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:54:10.809-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err Received malformed response from tcp connection vb=1. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f104/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:54:10.809-05:00 [INFO] Send batch count=219
2017/04/25 13:54:10 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 7 successful
CapiNozzle 2017-04-25T13:54:10.860-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:10.861-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:10.866-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:54:10.894-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:54:10.921-05:00 [INFO] Send batch count=287
2017/04/25 13:54:10 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:54:10 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 27 successful
CapiNozzle 2017-04-25T13:54:11.301-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:54:11 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 135 successful
CapiNozzle 2017-04-25T13:54:11.336-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:54:11 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 43 successful
CapiNozzle 2017-04-25T13:54:11.488-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:11.502-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:54:11 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 272 successful
CapiNozzle 2017-04-25T13:54:11.698-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:11.721-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:54:11 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 84 successful
StatisticsManager 2017-04-25T13:54:11.750-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:54:11 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 283 successful
CheckpointManager 2017-04-25T13:54:11.800-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
CheckpointManager 2017-04-25T13:54:11.800-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:54:11 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:54:11 UPR_STREAMREQ for vb 194 successful
DcpNozzle 2017-04-25T13:54:11.912-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:54:11.930-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:54:11.933-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:11.968-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:54:12.047-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:12.047-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:54:12.047-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:54:12.047-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
ReplicationManager 2017-04-25T13:54:12.047-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
PipelineManager 2017-04-25T13:54:12.054-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:54:12.054-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:54:12.057-05:00 [INFO] err_list=[{"time":"2017-04-25T13:54:12.057195341-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:54:12.060-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:54:12.152-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:12.152-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:12.173-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:54:12.266-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:12.289-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:54:12.595-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:12.617-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
StatisticsManager 2017-04-25T13:54:12.715-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:54:12.716-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223 {"Errors": "[{\"time\":\"2017-04-25T13:54:12.057195341-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 250, "dcp_datach_length": 0, "dcp_dispatch_time": 9, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 217, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 1, "docs_received_from_dcp": 39836, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 217, "set_filtered": 0, "set_received_from_dcp": 39836, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:54:12.836-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:12.836-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:54:12.836-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:54:12.836-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:12.836-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:12.965-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:12.965-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:12.989-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:13.011-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:54:13.015-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:54:13.454-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:13.454-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:54:13.454-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:54:13.455-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:13.455-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:13.543-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:13.543-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:13.564-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:54:13.613-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:13.630-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:54:13.944-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:54:13.961-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:54:14.124-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:14.145-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:54:14.475-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:54:14.535-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:54:14.727-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:14.728-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:54:14.728-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:54:14.728-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:14.728-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:14.814-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:14.814-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:14.846-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:54:15.002-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:54:15.002-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:54:15.002-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:54:15.002-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:15.002-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:15.088-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:15.088-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:15.106-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:54:15.164-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:15.189-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:54:15.495-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:15.512-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:54:15.754-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:15.786-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:54:16.379-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:16.394-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:54:17.077-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:17.078-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:54:17.078-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:54:17.078-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:17.078-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:17.146-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:17.146-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:17.166-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:54:17.480-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:17.480-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:54:17.480-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:54:17.480-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:17.480-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:17.556-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:17.557-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:17.577-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:54:17.835-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:17.850-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:54:18.265-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:18.286-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:54:18.494-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:18.515-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:54:18.691-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:18.702-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:54:19.222-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:19.222-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:54:19.222-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:54:19.222-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:19.222-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:19.332-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:19.332-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:19.343-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:54:19.523-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:19.523-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:54:19.523-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:54:19.523-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:19.523-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:19.574-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:19.574-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:19.585-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
GenericSupervisor 2017-04-25T13:54:19.703-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:54:19.703-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:54:19.703-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:54:19.750-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:54:19.763-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:54:19.907-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:19.918-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:54:20.170-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:54:20.182-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
ToplogyChangeDetector 2017-04-25T13:54:20.254-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:54:20.375-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:20.385-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:54:20.752-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:54:20.752-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:54:20.752-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:54:20.752-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:20.752-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:20.821-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:20.821-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:20.832-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:54:21.107-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:21.107-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:54:21.107-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:54:21.107-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:21.107-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:21.199-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:21.199-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:21.209-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
CapiNozzle 2017-04-25T13:54:21.290-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:21.301-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
CapiNozzle 2017-04-25T13:54:21.605-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:54:21.616-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:54:21.832-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:21.844-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
PipelineManager 2017-04-25T13:54:22.060-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:54:22.060-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:54:22.060-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:22.060-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
PipelineRuntimeCtx 2017-04-25T13:54:22.060-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:54:22.060-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:22.060-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:54:22.060-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:22.060563237 -0500 CDT
GenericSupervisor 2017-04-25T13:54:22.060-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:22.060-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:54:22.060-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:22.060-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:54:22.060-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:22.060-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:54:22.060-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:54:22.060-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:54:22.060-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:54:22.060-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-651181223 has been stopped
GenericPipeline 2017-04-25T13:54:22.061-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-651181223 is stopped
PipelineManager 2017-04-25T13:54:22.061-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:54:22.061-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:12.057195341-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:54:22.061-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:22.061-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
GenericPipeline 2017-04-25T13:54:22.061-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
GenericPipeline 2017-04-25T13:54:22.061-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
GenericPipeline 2017-04-25T13:54:22.061-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-651181223
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:22.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:22.062-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:54:22.062-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:22.062-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:54:22.062-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:22.062-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:54:22.062-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:22.062-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:54:22.062-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:22.062-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:22.062-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:54:22.062-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:22.062530689 -0500 CDT
DcpNozzle 2017-04-25T13:54:22.062-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:54:22.062-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:22.062779974 -0500 CDT
CapiNozzle 2017-04-25T13:54:22.063-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:54:22.063-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:54:22.063-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:54:22.063-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:54:22.063-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:54:22.063-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:54:22.063-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:22.064-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:22.064-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:54:22.064-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:22.064-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:54:22.066-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:22.066-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:22.066-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:22.066031262 -0500 CDT
DcpNozzle 2017-04-25T13:54:22.066-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:22.066042012 -0500 CDT
DcpNozzle 2017-04-25T13:54:22.066-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:54:22.066-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:54:22.066-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:22.066-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:54:22.066-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:22.066-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:54:22 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:54:22 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:54:22.709-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:22.710-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:54:22.710-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:54:22.764-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:22.765-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:54:22.765-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:22.765-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:54:22.765-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:22.765-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:54:22.765-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:54:22.765-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:54:22.766-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
GenericSupervisor 2017-04-25T13:54:22.768-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:54:22.768-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
GenericSupervisor 2017-04-25T13:54:22.768-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:22.768-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:22.768-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:22.768-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:22.768-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:54:22.768-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
CheckpointManager 2017-04-25T13:54:22.768-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
CapiUtils 2017-04-25T13:54:22.800-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:22.800-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:22.800-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:54:22.801-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
StatisticsManager 2017-04-25T13:54:22.802-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:54:22.802-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
XDCRFactory 2017-04-25T13:54:22.802-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:54:22.802-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456 is constructed, start it
GenericPipeline 2017-04-25T13:54:22.804-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[75 150 16 0 85 109 7 39 81 36 95 136 18 26 77 83 86 28 44 55 137 154 8 15 25 79 96 133 58 131 157 167 169 61 110 168 69 145 111 29 128 170 93 17 72 116 166 57 59 121 47 53 54 99 124 62 66 158 160 30 104 122 146 3 107 115 21 92 101 148 164 90 45 52 63 94 112 153 48 108 114 65 98 140 37 82 97 159 123 46 91 84 162 2 80 6 49 50 70 51 78 147 4 5 14 76 149 40 143 155 165 68 129 151 20 33 9 22 130 23 19 126 117 35 74 88 135 102 120 125 31 41 127 144 141 73 118 13 67 132 11 42 119 142 10 24 71 89 106 138 139 1 60 87 100 156 43 105 163 27 34 56 134 161 12 32 103 113 152 38 64] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[179 260 339 223 235 288 172 229 249 171 262 308 270 247 286 292 307 327 202 228 178 224 337 295 303 322 298 271 187 222 234 244 266 268 196 215 275 326 332 195 198 243 201 254 284 189 233 239 263 313 182 188 213 253 197 211 296 200 285 301 264 231 246 256 323 184 217 261 272 294 210 236 240 330 208 185 321 331 335 338 209 218 336 220 309 177 274 280 293 333 214 225 237 312 252 230 302 194 203 227 318 283 324 219 205 232 173 190 276 315 334 221 304 176 181 281 207 278 174 251 269 180 204 310 248 259 290 273 289 212 242 305 258 282 314 329 191 206 319 316 241 265 175 193 216 238 267 340 186 299 320 328 257 325 255 277 226 306 311 300 287 291 297 245 317 279 183 192 199 250]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[220 309 280 293 177 274 214 225 237 312 333 252 194 203 227 230 302 318 219 283 324 173 190 205 232 276 315 334 221 304 176 181 281 207 278 174 180 204 251 269 310 248 259 290 212 242 273 289 305 314 258 282 329 206 319 191 316 265 241 216 238 267 340 175 193 186 299 320 328 257 325 226 255 277 306 311 287 300 245 291 297 317 183 192 199 250 279 179 260 339 235 288 223 172 229 249 171 262 308 270 292 307 247 286 202 327 228 178 224 337 295 298 303 322 187 222 234 271 244 266 268 215 275 196 195 198 243 326 332 201 254 189 233 239 284 182 188 213 263 313 253 211 296 197 200 285 301 231 246 256 264 184 217 261 323 210 236 240 272 294 330 208 321 331 335 338 185 209 218 336] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[84 162 2 80 50 70 6 49 147 51 78 14 76 149 4 5 155 165 40 143 151 68 129 20 33 130 9 22 23 19 126 117 88 135 35 74 125 102 120 127 144 31 41 141 73 118 13 67 132 119 142 11 42 71 10 24 138 139 89 106 87 100 156 1 60 163 43 105 56 134 161 27 34 103 113 152 12 32 38 64 75 150 16 109 0 85 81 7 39 136 36 95 77 83 86 18 26 55 28 44 137 154 25 79 96 133 8 15 157 167 169 58 131 168 61 110 69 145 111 170 29 128 93 116 166 17 72 121 57 59 54 99 124 47 53 158 160 62 66 122 146 30 104 115 3 107 101 148 164 21 92 90 63 94 112 153 45 52 114 48 108 140 65 98 97 37 82 159 123 46 91]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info stats_interval:1000 filter_expression: active:true checkpoint_interval:1800 failure_restart_interval:10 optimistic_replication_threshold:256 replication_type:capi worker_batch_size:500 doc_batch_size_kb:2048]
CheckpointManager 2017-04-25T13:54:22.805-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456...
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:54:22.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:54:22.814-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:54:22.814-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:54:22.814-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:22.814-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:54:22.814-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:54:22.814-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:54:22.814-05:00 [INFO] Checkpointing starts in 1801.313 sec
PipelineRuntimeCtx 2017-04-25T13:54:22.814-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:54:22.815-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:54:22.815-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:54:22.815-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:54:22.816-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:54:22.816-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:54:22.817-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:54:22.841-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:54:22.841-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:54:22.845-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:54:22.854-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:54:22.864-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:54:22.864-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:54:22.880-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:22.880-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:22.880-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:54:22.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:54:22.895-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:54:22.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:54:22.899-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:22.899-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:54:22.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:54:22.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:54:22.899-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:54:22.927-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:54:22.927-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:54:22.927-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:54:22.954-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:54:22.954-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:54:23.015-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:54:23.091-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:54:23.091-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:23.091-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:54:23.123-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:54:23.123-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:54:23.123-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:54:23.123-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456 is started----------
PipelineManager 2017-04-25T13:54:23.124-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:54:23.124-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:54:23.124-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:54:23.895-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:54:24.581-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:54:24.581-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:54:24.581-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:54:24.581-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:54:24 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:54:24.627-05:00 [INFO] Send batch count=85
CapiNozzle 2017-04-25T13:54:24.662-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:24.662-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:54:24 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:54:24.676-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:54:24.678-05:00 [INFO] Send batch count=56
CapiNozzle 2017-04-25T13:54:24.692-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:54:24.695-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:24.695-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:54:24 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:54:24.720-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:54:24 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:54:24.824-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:24.835-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:54:24 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 315 successful
CapiNozzle 2017-04-25T13:54:24.874-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
StatisticsManager 2017-04-25T13:54:24.877-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:54:24.890-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:54:24 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:54:24 UPR_STREAMREQ for vb 224 successful
CapiNozzle 2017-04-25T13:54:24.998-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:25.012-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:54:25 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 30 successful
CapiNozzle 2017-04-25T13:54:25.050-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:54:25 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 234 successful
CapiNozzle 2017-04-25T13:54:25.086-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:54:25 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 338 successful
CapiNozzle 2017-04-25T13:54:25.143-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:25.143-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:54:25.143-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:54:25.143-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:54:25.144-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:54:25.156-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:54:25.157-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:54:25.163-05:00 [INFO] err_list=[{"time":"2017-04-25T13:54:25.163535285-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:54:25.163-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:54:25.214-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:25.214-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:54:25.214-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:54:25.214-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:25.214-05:00 [INFO] Send batch count=500
2017/04/25 13:54:25 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 49 successful
CapiNozzle 2017-04-25T13:54:25.244-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:25.245-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:54:25 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 244 successful
PipelineManager 2017-04-25T13:54:25.279-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:25.163535285-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
CapiNozzle 2017-04-25T13:54:25.282-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:25.282-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:25.288-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:54:25.317-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:54:25 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 87 successful
CapiNozzle 2017-04-25T13:54:25.640-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:54:25 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 187 successful
CapiNozzle 2017-04-25T13:54:25.682-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:54:25 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:54:25.744-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
CheckpointManager 2017-04-25T13:54:25.749-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:54:25 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 293 successful
StatisticsManager 2017-04-25T13:54:25.822-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:54:25.822-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456 {"Errors": "[{\"time\":\"2017-04-25T13:54:25.163535285-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 64, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 6812, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 64, "set_filtered": 0, "set_received_from_dcp": 6812, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

2017/04/25 13:54:25 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:54:25 UPR_STREAMREQ for vb 198 successful
CapiNozzle 2017-04-25T13:54:25.831-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:54:25 UPR_STREAMREQ for vb 99 successful
CapiNozzle 2017-04-25T13:54:25.841-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
DcpNozzle 2017-04-25T13:54:25.902-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:54:25.934-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:54:26.087-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:26.141-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:54:26.270-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:26.292-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:54:26.460-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:26.460-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:54:26.460-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:54:26.461-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:26.461-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:26.535-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:26.536-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:26.552-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:54:26.672-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:26.672-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:54:26.672-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:54:26.672-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:26.672-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:26.752-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:26.752-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:26.772-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:54:26.964-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:26.991-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:54:27.113-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:27.138-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:54:27.370-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:27.393-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:54:27.852-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:27.879-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:54:28.142-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:28.142-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:54:28.142-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:54:28.142-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:28.142-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:28.234-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:28.234-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:28.250-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:54:28.285-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:28.285-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:54:28.285-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:54:28.285-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:28.285-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:28.349-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:28.350-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:28.371-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:54:28.852-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:29.664-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:54:29.668-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:29.689-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:54:30.264-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:30.289-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:54:30.532-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:30.546-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:54:30.955-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:30.955-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:54:30.955-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:54:30.955-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:30.955-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:31.055-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:31.055-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:31.069-05:00 [INFO] Retrying update docs for vb 107 for the 1th time
CapiNozzle 2017-04-25T13:54:31.223-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:31.223-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:54:31.223-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:54:31.223-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:31.223-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:31.279-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:31.279-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:31.307-05:00 [INFO] Retrying update docs for vb 209 for the 1th time
CapiNozzle 2017-04-25T13:54:31.476-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:54:31.499-05:00 [INFO] Retrying update docs for vb 107 for the 2th time
CapiNozzle 2017-04-25T13:54:31.598-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:54:31.612-05:00 [INFO] Retrying update docs for vb 209 for the 2th time
CapiNozzle 2017-04-25T13:54:31.850-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:54:31.866-05:00 [INFO] Retrying update docs for vb 107 for the 3th time
CapiNozzle 2017-04-25T13:54:32.130-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:54:32.153-05:00 [INFO] Retrying update docs for vb 209 for the 3th time
CapiNozzle 2017-04-25T13:54:32.388-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:54:32.388-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 107 after 3 retries
CapiNozzle 2017-04-25T13:54:32.388-05:00 [ERROR] Raise error condition batch update docs failed for vb 107 after 3 retries
GenericSupervisor 2017-04-25T13:54:32.388-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:32.388-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:32.500-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:32.500-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:32.515-05:00 [INFO] Retrying update docs for vb 5 for the 1th time
CapiNozzle 2017-04-25T13:54:32.637-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:54:32.637-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 209 after 3 retries
CapiNozzle 2017-04-25T13:54:32.637-05:00 [ERROR] Raise error condition batch update docs failed for vb 209 after 3 retries
GenericSupervisor 2017-04-25T13:54:32.637-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:32.637-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:32.714-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:32.714-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:32.735-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
GenericSupervisor 2017-04-25T13:54:32.826-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:54:32.826-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:54:32.826-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:54:32.907-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:32.928-05:00 [INFO] Retrying update docs for vb 5 for the 2th time
ToplogyChangeDetector 2017-04-25T13:54:33.061-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:54:33.188-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:54:33.199-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:54:33.329-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:33.340-05:00 [INFO] Retrying update docs for vb 5 for the 3th time
CapiNozzle 2017-04-25T13:54:33.532-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:54:33.543-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:54:33.764-05:00 [ERROR] batchUpdateDocs for vb 5 failed with err receieved unexpected status code, 500, from update docs request for vb 5
.
CapiNozzle 2017-04-25T13:54:33.764-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 5 after 3 retries
CapiNozzle 2017-04-25T13:54:33.764-05:00 [ERROR] Raise error condition batch update docs failed for vb 5 after 3 retries
GenericSupervisor 2017-04-25T13:54:33.764-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:33.764-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:33.856-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:33.856-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:33.867-05:00 [INFO] Retrying update docs for vb 6 for the 1th time
CapiNozzle 2017-04-25T13:54:34.001-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:54:34.001-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:54:34.001-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:54:34.001-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:34.001-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:34.096-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:34.096-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:34.107-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:54:34.158-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:54:34.170-05:00 [INFO] Retrying update docs for vb 6 for the 2th time
CapiNozzle 2017-04-25T13:54:34.567-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:34.578-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:54:34.714-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:54:34.724-05:00 [INFO] Retrying update docs for vb 6 for the 3th time
CapiNozzle 2017-04-25T13:54:34.971-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:34.982-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:54:35.139-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:54:35.139-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 6 after 3 retries
CapiNozzle 2017-04-25T13:54:35.139-05:00 [ERROR] Raise error condition batch update docs failed for vb 6 after 3 retries
GenericSupervisor 2017-04-25T13:54:35.139-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:35.139-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:54:35.163-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:54:35.163-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:54:35.163-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:35.163-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
PipelineRuntimeCtx 2017-04-25T13:54:35.163-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:54:35.164-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:54:35.164-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:35.164-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:54:35.164-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:35.164109399 -0500 CDT
GenericSupervisor 2017-04-25T13:54:35.164-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:35.164-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:54:35.164-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:35.164-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:54:35.164-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:35.164-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:54:35.164-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:54:35.164-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:54:35.164-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-765596456 has been stopped
GenericPipeline 2017-04-25T13:54:35.164-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-765596456 is stopped
PipelineManager 2017-04-25T13:54:35.164-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:54:35.164-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:54:35.164-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
PipelineManager 2017-04-25T13:54:35.164-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:25.163535285-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

GenericPipeline 2017-04-25T13:54:35.164-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
GenericPipeline 2017-04-25T13:54:35.164-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
PipelineManager 2017-04-25T13:54:35.164-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:35.164-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-765596456
AsyncListener 2017-04-25T13:54:35.164-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:35.164-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:35.164-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:35.164-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:35.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:54:35.165-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:35.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:35.165-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:54:35.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:35.165326701 -0500 CDT
DcpNozzle 2017-04-25T13:54:35.165-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:35.165601776 -0500 CDT
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.165-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:35.165-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:35.166-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:35.166-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:54:35.166-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:54:35.166-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:35.166-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:54:35.167-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:35.167-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:35.167846538 -0500 CDT
DcpNozzle 2017-04-25T13:54:35.167-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:54:35.167-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:35.167-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:54:35.168-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:35.168-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:35.168050517 -0500 CDT
DcpNozzle 2017-04-25T13:54:35.168-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:54:35.168-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:35.168-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:54:35 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:54:35 Stream Closed for vb 0 StreamEnd simulated
CapiNozzle 2017-04-25T13:54:35.214-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:54:35.214-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:54:35.214-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:35.214-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:54:35.298-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:35.298-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:54:35.298-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:54:35.344-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:35.344-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:54:35.344-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:35.344-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:54:35.345-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:35.345-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:54:35.345-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:54:35.345-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:54:35.345-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
GenericSupervisor 2017-04-25T13:54:35.347-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:54:35.347-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
GenericSupervisor 2017-04-25T13:54:35.347-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:35.347-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:35.347-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:35.347-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:35.347-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:54:35.347-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
CheckpointManager 2017-04-25T13:54:35.347-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
CapiUtils 2017-04-25T13:54:35.376-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:35.376-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:35.376-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:54:35.377-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
StatisticsManager 2017-04-25T13:54:35.377-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:54:35.378-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
XDCRFactory 2017-04-25T13:54:35.378-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:54:35.378-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159 is constructed, start it
GenericPipeline 2017-04-25T13:54:35.379-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[101 145 29 52 103 63 100 112 129 133 2 43 82 104 120 130 28 118 105 9 54 166 169 72 90 75 57 60 79 109 4 77 132 135 126 96 99 80 16 71 76 155 14 55 125 94 137 159 46 91 58 161 1 50 121 123 131 165 39 95 69 97 115 22 49 160 31 89 48 86 78 147 10 68 83 7 34 45 59 148 24 37 56 127 23 36 33 81 150 61 106 11 30 113 153 13 47 117 51 85 64 154 40 87 110 6 108 67 70 0 17 162 163 134 158 143 73 170 98 102 142 32 93 21 107 124 151 5 66 138 149 156 8 15 74 53 42 84 122 3 19 20 65 167 35 139 27 88 119 152 44 116 140 146 164 18 26 25 136 144 41 128 92 141 62 111 114 157 168 12 38] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[232 260 321 327 338 212 222 247 334 186 180 218 267 300 177 283 179 187 204 210 314 183 258 273 194 229 269 284 309 315 249 272 202 337 192 196 251 328 213 234 282 244 220 199 230 293 324 275 303 257 250 256 294 336 261 335 233 245 299 306 259 320 227 295 310 329 255 313 253 208 277 182 288 276 274 278 238 270 225 228 243 188 291 312 197 205 340 215 231 264 319 173 268 285 308 331 246 265 298 193 200 206 271 279 323 326 195 332 333 248 307 190 198 266 289 185 287 325 178 184 252 214 203 216 302 316 174 242 281 209 305 318 223 241 221 280 239 172 290 297 175 235 191 201 207 219 262 304 339 211 226 301 171 181 322 217 296 236 240 263 311 176 224 292 330 189 237 254 286 317]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[53 84 122 3 19 42 20 65 35 139 167 27 88 119 152 116 140 146 164 18 26 44 25 136 41 128 144 92 62 111 141 114 157 168 12 38 101 145 29 52 103 100 112 129 133 2 43 63 82 104 130 28 118 120 9 54 105 169 72 90 166 57 60 75 109 4 77 79 132 135 96 99 126 16 71 80 155 14 55 76 125 137 159 46 91 94 58 1 50 161 123 131 165 39 95 121 97 115 22 49 69 31 89 160 48 86 78 147 10 83 7 34 68 59 148 24 37 45 127 23 36 56 33 81 61 106 150 11 30 113 153 13 47 51 85 117 64 154 40 87 6 108 110 70 0 17 67 163 134 158 162 143 73 170 102 142 32 93 98 21 107 151 5 66 124 149 156 8 15 138 74] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[239 280 172 290 297 175 235 201 207 219 262 304 191 339 226 301 211 181 171 322 296 217 236 240 263 311 176 292 330 224 237 254 286 317 189 260 321 327 338 232 222 247 334 186 212 218 267 300 180 177 179 187 283 210 204 314 183 258 273 194 229 269 284 309 315 249 272 202 337 192 196 251 328 213 234 282 220 244 199 230 293 324 303 257 275 250 256 294 336 261 335 233 245 299 306 259 320 295 310 329 227 313 253 255 277 182 208 276 288 278 238 270 274 225 228 243 188 291 312 205 340 197 215 231 264 319 268 285 308 331 173 265 298 246 200 206 193 271 279 323 326 332 333 195 307 248 190 198 266 289 185 287 325 178 252 184 214 216 302 316 203 174 242 281 209 305 318 241 221 223]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: active:true worker_batch_size:500 optimistic_replication_threshold:256 stats_interval:1000 log_level:Info replication_type:capi checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:54:35.379-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159...
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:35.387-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:54:35.387-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:35.387-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:54:35.387-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:54:35.387-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:54:35.387-05:00 [INFO] Checkpointing starts in 1801.69 sec
PipelineRuntimeCtx 2017-04-25T13:54:35.387-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:54:35.387-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:54:35.388-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:54:35.388-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:54:35.390-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:54:35.390-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:54:35.390-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:54:35.390-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:54:35.392-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:54:35.436-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:54:35.436-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiUtils 2017-04-25T13:54:35.439-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:35.439-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:35.439-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:54:35.439-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
ToplogyChangeDetector 2017-04-25T13:54:35.449-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:54:35.458-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:54:35.458-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:54:35.468-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:54:35.475-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:54:35.512-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:54:35.512-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:35.512-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:54:35.512-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:54:35.512-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:54:35.513-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:54:35.547-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:54:35.547-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:54:35.547-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
CheckpointManager 2017-04-25T13:54:35.559-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:54:35.564-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:54:35.564-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:54:35.682-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:54:35.682-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:35.682-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:54:35.700-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:54:35.700-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:54:35.701-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:54:35.701-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159 is started----------
PipelineManager 2017-04-25T13:54:35.701-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:54:35.701-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:54:35.701-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:54:35.885-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:54:35.885-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:54:35.885-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:54:35.885-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:54:35 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:54:35 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:54:35.959-05:00 [INFO] Send batch count=25
CapiNozzle 2017-04-25T13:54:35.966-05:00 [ERROR] Received malformed response from tcp connection vb=300. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f6/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}

CapiNozzle 2017-04-25T13:54:35.966-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err Received malformed response from tcp connection vb=300. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f6/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}
.
CapiNozzle 2017-04-25T13:54:35.977-05:00 [INFO] Send batch count=146
CapiNozzle 2017-04-25T13:54:35.984-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:54:36 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 205 successful
CapiNozzle 2017-04-25T13:54:36.040-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:36.040-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:54:36 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 11 successful
CapiNozzle 2017-04-25T13:54:36.066-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:36.075-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:54:36.097-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
2017/04/25 13:54:36 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 210 successful
CapiNozzle 2017-04-25T13:54:36.144-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:54:36 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 20 successful
CapiNozzle 2017-04-25T13:54:36.188-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
2017/04/25 13:54:36 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 37 successful
CapiNozzle 2017-04-25T13:54:36.352-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:36.358-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:36.358-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:54:36.358-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:54:36.358-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:54:36.358-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
CapiNozzle 2017-04-25T13:54:36.367-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
PipelineManager 2017-04-25T13:54:36.378-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:54:36.378-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:54:36.380-05:00 [INFO] err_list=[{"time":"2017-04-25T13:54:36.380685097-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
PipelineManager 2017-04-25T13:54:36.380-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:54:36.398-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:54:36.399-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159 {"Errors": "[{\"time\":\"2017-04-25T13:54:36.380685097-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]", "Status": "Pending"}

2017/04/25 13:54:36 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 246 successful
CapiNozzle 2017-04-25T13:54:36.475-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:36.476-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:36.502-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:54:36 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 158 successful
CapiNozzle 2017-04-25T13:54:36.660-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:36.685-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
2017/04/25 13:54:36 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 90 successful
CapiNozzle 2017-04-25T13:54:36.891-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:54:36.931-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:54:36 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:54:36 UPR_STREAMREQ for vb 98 successful
CheckpointManager 2017-04-25T13:54:36.979-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
CheckpointManager 2017-04-25T13:54:36.980-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:54:37.006-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:37.006-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:54:37.006-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:54:37.006-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:37.006-05:00 [INFO] Send batch count=500
2017/04/25 13:54:37 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:54:37 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:54:37 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:54:37 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:54:37.021-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:54:37.098-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:37.098-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
DcpNozzle 2017-04-25T13:54:37.105-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:54:37.124-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:54:37.286-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:54:37.304-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:54:37.490-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:37.501-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:54:37.691-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:54:37.691-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:54:37.691-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:54:37.691-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:37.692-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:37.771-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:37.771-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:37.789-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:54:37.921-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:37.939-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:54:38.101-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:38.115-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:54:38.338-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:38.338-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:54:38.338-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:54:38.338-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:38.338-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:38.433-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:38.433-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:38.453-05:00 [INFO] Retrying update docs for vb 3 for the 1th time
CapiNozzle 2017-04-25T13:54:38.537-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:38.561-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:54:38.776-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:54:38.802-05:00 [INFO] Retrying update docs for vb 3 for the 2th time
CapiNozzle 2017-04-25T13:54:38.940-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:38.940-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:54:38.940-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:54:38.940-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:38.940-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:39.024-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:39.025-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:39.049-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:54:39.258-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:54:39.276-05:00 [INFO] Retrying update docs for vb 3 for the 3th time
CapiNozzle 2017-04-25T13:54:39.455-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:39.477-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
CapiNozzle 2017-04-25T13:54:39.626-05:00 [ERROR] batchUpdateDocs for vb 3 failed with err receieved unexpected status code, 500, from update docs request for vb 3
.
CapiNozzle 2017-04-25T13:54:39.626-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 3 after 3 retries
CapiNozzle 2017-04-25T13:54:39.626-05:00 [ERROR] Raise error condition batch update docs failed for vb 3 after 3 retries
GenericSupervisor 2017-04-25T13:54:39.626-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:39.626-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:39.688-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:39.689-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:39.706-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:54:39.858-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:39.874-05:00 [INFO] Retrying update docs for vb 301 for the 3th time
CapiNozzle 2017-04-25T13:54:39.945-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:39.960-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
PipelineManager 2017-04-25T13:54:40.255-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:36.380685097-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:54:40.264-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:54:40.264-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 301 after 3 retries
CapiNozzle 2017-04-25T13:54:40.264-05:00 [ERROR] Raise error condition batch update docs failed for vb 301 after 3 retries
GenericSupervisor 2017-04-25T13:54:40.264-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:40.264-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:40.429-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
ReplicationManager 2017-04-25T13:54:40.469-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:54:40.469-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:54:40.473-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:40.473-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:40.487-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:54:40.840-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:54:40.840-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:54:40.840-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:54:40.840-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:40.840-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:40.948-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:40.948-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:40.960-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:54:41.394-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:41.420-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:54:41.872-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:41.896-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:54:42.401-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:42.415-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:54:42.738-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:42.751-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:54:43.002-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:54:43.002-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:54:43.002-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:54:43.002-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:43.002-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:43.064-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:43.064-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:43.092-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:54:43.190-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:43.190-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:54:43.190-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:54:43.190-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:43.191-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:43.296-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:43.296-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:43.316-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:54:43.337-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:54:43.361-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:54:44.601-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:44.616-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:54:44.959-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:54:44.970-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:54:45.195-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:45.213-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
GenericSupervisor 2017-04-25T13:54:45.393-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:54:45.393-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:54:45.393-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:54:45.420-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:54:45.420-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:54:45.420-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:54:45.420-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:45.420-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:45.560-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:45.560-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:45.571-05:00 [INFO] Retrying update docs for vb 204 for the 1th time
ToplogyChangeDetector 2017-04-25T13:54:45.623-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:54:45.693-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:45.693-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:54:45.693-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:54:45.693-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:45.693-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:45.770-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:45.770-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:45.780-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:54:46.051-05:00 [ERROR] batchUpdateDocs for vb 204 failed with err receieved unexpected status code, 500, from update docs request for vb 204
.
CapiNozzle 2017-04-25T13:54:46.062-05:00 [INFO] Retrying update docs for vb 204 for the 2th time
CapiNozzle 2017-04-25T13:54:46.219-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:46.230-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
PipelineManager 2017-04-25T13:54:46.380-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:54:46.381-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:54:46.381-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:46.381-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
PipelineRuntimeCtx 2017-04-25T13:54:46.381-05:00 [INFO] Pipeline context is stopping...
ToplogyChangeDetector 2017-04-25T13:54:46.381-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:54:46.381-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:46.381-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:54:46.381-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:46.381283871 -0500 CDT
GenericSupervisor 2017-04-25T13:54:46.381-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:46.381-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:54:46.381-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:46.381-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:54:46.381-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:46.381-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:54:46.381-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:54:46.381-05:00 [INFO] StatisticsManager Stopped
PipelineRuntimeCtx 2017-04-25T13:54:46.381-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-345109159 has been stopped
GenericPipeline 2017-04-25T13:54:46.381-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-345109159 is stopped
PipelineManager 2017-04-25T13:54:46.381-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:54:46.381-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:36.380685097-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:54:46.382-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-345109159
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:54:46.382-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:46.382-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:54:46.382-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:54:46.382-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:46.382-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.382-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:46.382-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:54:46.382-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:54:46.383-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:46.383-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:54:46.383-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:46.383-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:46.382851771 -0500 CDT
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:54:46.383-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:46.383-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:46.383144155 -0500 CDT
CapiNozzle 2017-04-25T13:54:46.383-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:54:46.384-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:54:46.384-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:54:46.384-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:46.384-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:54:46.384-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:54:46.384-05:00 [INFO] Ask uprfeed to close
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:54:46.384-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:46.384801981 -0500 CDT
DcpNozzle 2017-04-25T13:54:46.384-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:46.384-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:46.384-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:46.385-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:54:46.384-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:46.385-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:54:46.387-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:46.387-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:46.387941846 -0500 CDT
DcpNozzle 2017-04-25T13:54:46.387-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:54:46.388-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:46.388-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:54:46 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:54:46 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:54:46 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:54:46 Stream Closed for vb 2 StreamEnd simulated
XDCRFactory 2017-04-25T13:54:46.512-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:46.512-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:54:46.512-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:54:46.587-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:46.588-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:54:46.588-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:46.588-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:54:46.588-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:46.588-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:54:46.588-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:54:46.589-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:54:46.589-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
GenericSupervisor 2017-04-25T13:54:46.591-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:54:46.591-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
GenericSupervisor 2017-04-25T13:54:46.591-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:46.591-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:46.591-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:46.591-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:46.591-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:54:46.591-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
CheckpointManager 2017-04-25T13:54:46.591-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
CapiUtils 2017-04-25T13:54:46.619-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:46.619-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:46.619-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:54:46.620-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
StatisticsManager 2017-04-25T13:54:46.620-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:54:46.620-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
XDCRFactory 2017-04-25T13:54:46.620-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:54:46.621-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867 is constructed, start it
GenericPipeline 2017-04-25T13:54:46.622-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[322 331 197 213 226 259 309 182 189 263 291 220 238 172 208 209 308 287 288 184 217 218 284 255 179 227 240 274 306 329 225 195 202 249 316 233 176 296 276 181 212 222 248 297 216 320 178 183 264 321 280 221 278 305 196 198 294 333 283 177 187 191 312 174 295 206 253 315 290 205 273 193 219 262 310 180 230 236 267 298 175 282 286 293 210 251 275 258 200 215 300 204 304 318 326 338 261 214 246 211 201 231 250 279 319 340 185 234 302 229 260 332 243 303 272 247 244 317 199 232 271 311 334 268 269 299 330 186 324 301 194 328 190 188 323 171 239 289 292 228 313 257 285 266 335 314 265 270 281 336 254 237 242 307 252 277 337 203 223 325 256 192 207 224 235 245 173 339 241 327] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[120 146 63 84 106 57 82 139 62 168 156 68 95 154 80 87 122 153 155 46 55 77 157 161 74 160 165 0 92 109 130 158 167 30 39 50 118 65 97 103 32 134 163 22 25 42 53 119 143 136 140 15 76 86 148 149 162 169 99 113 141 2 5 29 45 129 21 137 150 4 142 147 16 40 54 70 90 49 81 94 108 166 116 133 1 24 33 23 9 19 88 12 85 89 7 96 135 111 164 58 67 152 17 114 123 83 20 38 56 131 170 47 79 128 91 10 35 66 8 73 3 13 145 100 14 48 52 27 61 78 144 93 159 31 41 110 6 44 71 11 104 124 138 105 132 37 43 115 18 69 26 72 102 101 117 121 51 64 75 34 60 151 59 112 125 28 36 98 107 127 126]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[12 85 89 7 96 135 111 164 58 67 17 114 123 152 20 38 56 83 47 79 128 131 170 10 35 66 91 8 73 3 13 145 14 48 52 100 27 61 78 144 93 159 31 41 6 44 71 110 11 104 124 138 105 132 37 43 115 18 69 26 72 102 101 117 121 51 64 75 34 60 59 112 125 151 28 36 98 107 127 126 63 84 106 120 146 57 82 139 62 168 68 95 154 156 153 155 46 55 77 80 87 122 157 161 74 160 165 0 92 109 30 39 50 130 158 167 65 97 103 118 32 134 163 22 25 42 53 119 143 15 76 86 136 140 169 99 113 141 148 149 162 2 5 29 45 129 21 137 4 142 147 150 16 40 54 70 90 49 81 94 108 166 1 24 33 116 133 23 9 19 88] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[290 205 273 310 180 193 219 262 298 175 230 236 267 210 251 275 282 286 293 258 300 200 215 304 318 204 326 338 261 211 214 246 279 319 201 231 250 340 302 185 234 229 260 332 243 303 247 272 199 232 244 317 268 269 271 311 334 299 330 186 324 194 301 190 328 188 323 171 239 289 228 292 257 285 313 266 335 314 336 254 265 270 281 307 237 242 203 223 252 277 337 256 325 235 245 173 192 207 224 339 241 327 331 322 259 309 182 189 197 213 226 220 238 263 291 209 308 172 208 184 217 218 287 288 255 284 274 306 179 227 240 329 225 195 202 249 316 233 176 296 276 181 222 248 212 297 216 320 264 178 183 321 221 280 196 198 278 305 283 294 333 312 177 187 191 174 295 206 253 315]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi active:true worker_batch_size:500 failure_restart_interval:10 source_nozzle_per_node:2 log_level:Info]
CheckpointManager 2017-04-25T13:54:46.622-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867...
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:54:46.629-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
ToplogyChangeDetector 2017-04-25T13:54:46.629-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:54:46.629-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:54:46.629-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:46.629-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:54:46.629-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:54:46.629-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:54:46.629-05:00 [INFO] Checkpointing starts in 1801.771 sec
PipelineRuntimeCtx 2017-04-25T13:54:46.629-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:54:46.629-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:54:46.629-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:54:46.629-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:54:46.631-05:00 [INFO] Service StatisticsManager has been started
StatisticsManager 2017-04-25T13:54:46.631-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:54:46.634-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
ToplogyChangeDetector 2017-04-25T13:54:46.646-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:54:46.649-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:46.649-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:46.649-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:54:46.653-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:54:46.679-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:54:46.679-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:54:46.679-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:54:46.704-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:54:46.704-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
StatisticsManager 2017-04-25T13:54:46.709-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:54:46.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:54:46.743-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:46.743-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:54:46.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:54:46.743-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:54:46.744-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:54:46.759-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:54:46.759-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:54:46.759-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:54:46.794-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:54:46.794-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:54:46.820-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:54:46.914-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:54:46.914-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:46.914-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:54:46.954-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:54:46.954-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:54:46.955-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:54:46.955-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867 is started----------
PipelineManager 2017-04-25T13:54:46.955-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:54:46.955-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:54:46.955-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:54:46.995-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:54:46.996-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:54:46.996-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:54:46.996-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:54:47 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:54:47.083-05:00 [INFO] Send batch count=19
CapiNozzle 2017-04-25T13:54:47.096-05:00 [ERROR] Received malformed response from tcp connection vb=1. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f204/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:54:47.096-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err Received malformed response from tcp connection vb=1. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f204/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
CapiNozzle 2017-04-25T13:54:47.133-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
2017/04/25 13:54:47 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 308 successful
CapiNozzle 2017-04-25T13:54:47.181-05:00 [INFO] Send batch count=1
2017/04/25 13:54:47 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:54:47.187-05:00 [INFO] Send batch count=67
CapiNozzle 2017-04-25T13:54:47.194-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:47.198-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:47.198-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:47.216-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:54:47.218-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
2017/04/25 13:54:47 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 133 successful
CapiNozzle 2017-04-25T13:54:47.287-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:47.301-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:54:47.364-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:54:47 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:54:47.405-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:54:47.414-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:47.414-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:54:47.414-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:54:47.414-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
ReplicationManager 2017-04-25T13:54:47.415-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]
PipelineManager 2017-04-25T13:54:47.429-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:54:47.429-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:54:47.432-05:00 [INFO] err_list=[{"time":"2017-04-25T13:54:47.432852954-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]
PipelineManager 2017-04-25T13:54:47.432-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:54:47 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 60 successful
CapiNozzle 2017-04-25T13:54:47.501-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:47.501-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:47.541-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:54:47.565-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:54:47 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 273 successful
CapiNozzle 2017-04-25T13:54:47.599-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
StatisticsManager 2017-04-25T13:54:47.641-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:54:47.642-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867 {"Errors": "[{\"time\":\"2017-04-25T13:54:47.432852954-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]", "Status": "Pending"}

2017/04/25 13:54:47 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 84 successful
CapiNozzle 2017-04-25T13:54:47.783-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:47.783-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:54:47.783-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:54:47.783-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:47.783-05:00 [INFO] Send batch count=500
2017/04/25 13:54:47 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 92 successful
CapiNozzle 2017-04-25T13:54:47.911-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:47.911-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err write tcp 10.202.98.33:9091: broken pipe.
CheckpointManager 2017-04-25T13:54:47.937-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
CheckpointManager 2017-04-25T13:54:47.937-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:54:47.938-05:00 [INFO] Retrying update docs for vb 208 for the 1th time
DcpNozzle 2017-04-25T13:54:47.949-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
2017/04/25 13:54:47 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:54:47 UPR_STREAMREQ for vb 98 successful
DcpNozzle 2017-04-25T13:54:48.063-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:54:48.101-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:48.119-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:54:48.411-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:54:48.433-05:00 [INFO] Retrying update docs for vb 208 for the 2th time
CapiNozzle 2017-04-25T13:54:48.668-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:48.700-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:54:49.115-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:54:49.127-05:00 [INFO] Retrying update docs for vb 208 for the 3th time
CapiNozzle 2017-04-25T13:54:49.313-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:54:49.313-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:54:49.313-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:54:49.313-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:49.313-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:49.461-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:49.461-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:49.487-05:00 [ERROR] batchUpdateDocs for vb 208 failed with err receieved unexpected status code, 500, from update docs request for vb 208
.
CapiNozzle 2017-04-25T13:54:49.487-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 208 after 3 retries
CapiNozzle 2017-04-25T13:54:49.487-05:00 [ERROR] Raise error condition batch update docs failed for vb 208 after 3 retries
GenericSupervisor 2017-04-25T13:54:49.487-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:49.487-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:49.489-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:54:49.561-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:49.561-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:49.597-05:00 [INFO] Retrying update docs for vb 209 for the 1th time
CapiNozzle 2017-04-25T13:54:49.917-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:49.928-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:54:50.172-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:54:50.191-05:00 [INFO] Retrying update docs for vb 209 for the 2th time
CapiNozzle 2017-04-25T13:54:50.535-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:50.559-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:54:50.723-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:54:50.740-05:00 [INFO] Retrying update docs for vb 209 for the 3th time
CapiNozzle 2017-04-25T13:54:50.899-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:54:50.899-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:54:50.899-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:54:50.899-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:50.899-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:51.038-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:51.044-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:51.052-05:00 [ERROR] batchUpdateDocs for vb 209 failed with err receieved unexpected status code, 500, from update docs request for vb 209
.
CapiNozzle 2017-04-25T13:54:51.052-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 209 after 3 retries
CapiNozzle 2017-04-25T13:54:51.052-05:00 [ERROR] Raise error condition batch update docs failed for vb 209 after 3 retries
GenericSupervisor 2017-04-25T13:54:51.052-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:51.052-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:51.067-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:54:51.110-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:51.110-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:51.129-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:54:51.488-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:51.519-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:54:51.605-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:51.624-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:54:51.996-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:52.011-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:54:52.161-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:52.210-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:54:52.407-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:54:52.407-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:54:52.407-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:54:52.407-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:52.407-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:52.500-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:52.500-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:52.519-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:54:52.603-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:54:52.603-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:54:52.603-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:54:52.603-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:52.603-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:52.699-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:52.700-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:52.729-05:00 [INFO] Retrying update docs for vb 212 for the 1th time
CapiNozzle 2017-04-25T13:54:52.874-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:52.902-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:54:53.267-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:54:53.283-05:00 [INFO] Retrying update docs for vb 212 for the 2th time
CapiNozzle 2017-04-25T13:54:53.478-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:53.510-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:54:53.637-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:54:53.659-05:00 [INFO] Retrying update docs for vb 212 for the 3th time
CapiNozzle 2017-04-25T13:54:53.809-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:54:53.809-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:54:53.809-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:54:53.809-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:53.809-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:53.895-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:53.895-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:53.916-05:00 [INFO] Retrying update docs for vb 103 for the 1th time
CapiNozzle 2017-04-25T13:54:54.004-05:00 [ERROR] batchUpdateDocs for vb 212 failed with err receieved unexpected status code, 500, from update docs request for vb 212
.
CapiNozzle 2017-04-25T13:54:54.004-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 212 after 3 retries
CapiNozzle 2017-04-25T13:54:54.004-05:00 [ERROR] Raise error condition batch update docs failed for vb 212 after 3 retries
GenericSupervisor 2017-04-25T13:54:54.004-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:54.004-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:54.118-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:54.118-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:54.142-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:54:55.022-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:54:55.037-05:00 [INFO] Retrying update docs for vb 103 for the 2th time
PipelineManager 2017-04-25T13:54:55.232-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:47.432852954-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries]}
]
CapiNozzle 2017-04-25T13:54:55.513-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:55.524-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:54:55.954-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:54:55.965-05:00 [INFO] Retrying update docs for vb 103 for the 3th time
CapiNozzle 2017-04-25T13:54:56.177-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:56.188-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:54:56.312-05:00 [ERROR] batchUpdateDocs for vb 103 failed with err receieved unexpected status code, 500, from update docs request for vb 103
.
CapiNozzle 2017-04-25T13:54:56.312-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 103 after 3 retries
CapiNozzle 2017-04-25T13:54:56.312-05:00 [ERROR] Raise error condition batch update docs failed for vb 103 after 3 retries
GenericSupervisor 2017-04-25T13:54:56.312-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:56.312-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:54:56.409-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:56.409-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:56.420-05:00 [INFO] Retrying update docs for vb 6 for the 1th time
CapiNozzle 2017-04-25T13:54:56.550-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:54:56.550-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:54:56.550-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:54:56.550-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:54:56.550-05:00 [INFO] Send batch count=500
GenericSupervisor 2017-04-25T13:54:56.630-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:54:56.630-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:54:56.630-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:54:56.649-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:56.649-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:54:56.660-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
ToplogyChangeDetector 2017-04-25T13:54:56.838-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:54:56.875-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:54:56.886-05:00 [INFO] Retrying update docs for vb 6 for the 2th time
CapiNozzle 2017-04-25T13:54:57.113-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:57.124-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:54:57.283-05:00 [ERROR] batchUpdateDocs for vb 6 failed with err receieved unexpected status code, 500, from update docs request for vb 6
.
CapiNozzle 2017-04-25T13:54:57.295-05:00 [INFO] Retrying update docs for vb 6 for the 3th time
PipelineManager 2017-04-25T13:54:57.433-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries 
PipelineManager 2017-04-25T13:54:57.433-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:54:57.433-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:54:57.433-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
PipelineRuntimeCtx 2017-04-25T13:54:57.433-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:54:57.433-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:57.433-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:54:57.433-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:57.433361197 -0500 CDT
GenericSupervisor 2017-04-25T13:54:57.433-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:57.433-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:54:57.433-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:57.433-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:54:57.433-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:54:57.433-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:54:57.433-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:54:57.433-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:54:57.433-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:54:57.433-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-588810867 has been stopped
GenericPipeline 2017-04-25T13:54:57.433-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-588810867 is stopped
PipelineManager 2017-04-25T13:54:57.433-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:54:57.434-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:54:47.432852954-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 1 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:54:57.434-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-588810867
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:54:57.434-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:54:57.434-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:57.434-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:54:57.434-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:54:57.434-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:54:57.434-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.434-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:54:57.434-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:54:57.435-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:57.435-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:57.435-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
DcpNozzle 2017-04-25T13:54:57.435-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.435-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
DcpNozzle 2017-04-25T13:54:57.435-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:54:57.435-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:57.434886033 -0500 CDT
CapiNozzle 2017-04-25T13:54:57.435-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:57.435171595 -0500 CDT
CapiNozzle 2017-04-25T13:54:57.435-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:54:57.436-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:54:57.436-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:54:57.436-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:54:57.436-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:54:57.436-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:54:57.436-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
GenericPipeline 2017-04-25T13:54:57.436-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:57.436-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:54:57.436-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:57.436-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:54:57.437-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:57.437-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:57.437750064 -0500 CDT
DcpNozzle 2017-04-25T13:54:57.437-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:54:57.437-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:57.437-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:54:57.438-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:54:57.438-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:54:57.438074482 -0500 CDT
DcpNozzle 2017-04-25T13:54:57.438-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:54:57.438-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:54:57.438-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:54:57 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:54:57 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:54:57 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:54:57 Stream Closed for vb 2 StreamEnd simulated
2017/04/25 13:54:57 Stream Closed for vb 3 StreamEnd simulated
XDCRFactory 2017-04-25T13:54:57.547-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:57.547-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:54:57.547-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:54:57.611-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:54:57.612-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:54:57.612-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:57.612-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:54:57.612-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:54:57.612-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:54:57.612-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:54:57.612-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:54:57.613-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
GenericSupervisor 2017-04-25T13:54:57.615-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:54:57.615-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
GenericSupervisor 2017-04-25T13:54:57.615-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:57.615-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:57.615-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:57.615-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:54:57.615-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:54:57.615-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
CheckpointManager 2017-04-25T13:54:57.615-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
CapiUtils 2017-04-25T13:54:57.642-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:57.642-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:57.642-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:54:57.643-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
StatisticsManager 2017-04-25T13:54:57.644-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:54:57.644-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
XDCRFactory 2017-04-25T13:54:57.644-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:54:57.644-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511 is constructed, start it
GenericPipeline 2017-04-25T13:54:57.645-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[301 334 203 220 231 290 258 270 275 279 179 240 305 311 326 237 321 176 184 209 269 292 329 303 228 251 323 232 267 282 320 264 225 302 316 188 174 189 222 300 306 201 238 252 319 336 298 173 177 171 190 276 296 338 206 313 215 197 259 198 242 248 249 223 272 193 205 207 245 318 186 213 253 288 312 199 230 236 289 331 182 317 229 283 309 310 322 211 254 257 332 218 295 325 278 293 307 250 294 224 239 214 268 196 284 202 287 183 281 227 261 277 216 263 315 333 314 200 256 304 219 262 339 180 192 217 172 181 273 191 221 247 274 285 210 337 335 212 260 271 185 233 280 291 327 328 255 265 178 208 241 187 195 204 266 308 175 194 234 297 330 246 243 286 299 324 226 235 244 340] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[8 165 144 49 86 113 149 12 26 41 92 103 150 87 2 28 89 105 128 132 143 23 64 91 11 118 126 159 46 116 130 97 101 114 117 134 25 55 170 10 21 79 69 73 88 96 136 31 145 162 6 18 33 50 63 14 34 59 44 151 42 158 40 83 133 160 13 24 1 99 57 85 138 164 140 167 39 52 62 112 156 157 71 84 102 90 169 3 54 100 120 125 154 27 70 77 166 110 131 147 161 163 5 15 17 29 127 129 108 109 139 168 56 60 111 95 98 122 153 0 20 146 137 141 4 32 7 51 148 115 36 106 155 142 19 68 75 107 53 72 94 152 30 45 123 135 37 58 80 67 9 22 61 78 81 119 38 43 48 76 124 35 65 104 121 66 47 16 93 74 82]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[336 298 171 173 177 190 276 296 338 206 313 197 215 198 242 248 259 249 193 205 207 223 272 245 186 213 253 288 318 199 230 236 289 312 182 317 331 229 283 309 211 254 257 310 322 332 218 295 325 278 293 307 224 239 250 294 214 268 196 284 202 287 183 227 261 277 281 216 263 315 333 200 256 304 314 219 262 339 180 192 217 172 181 273 191 221 247 274 285 210 337 335 212 260 271 185 233 280 291 327 328 255 265 178 208 241 187 195 204 266 308 175 194 234 297 330 246 243 286 299 324 226 235 244 340 203 220 231 290 301 334 258 270 179 240 275 279 305 311 326 176 237 321 184 209 269 292 329 303 228 251 323 232 267 282 320 264 225 302 316 188 174 189 222 300 306 201 238 252 319] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[10 21 79 69 73 88 96 136 31 145 162 6 18 33 50 63 14 34 59 44 151 42 158 40 83 133 160 13 24 1 99 57 85 138 164 140 39 52 62 112 156 157 167 71 84 102 90 3 54 100 120 125 154 169 27 70 77 166 110 131 147 161 5 15 17 29 127 129 163 108 109 139 168 56 60 111 95 98 122 153 0 20 146 137 141 4 32 7 51 148 115 36 106 155 142 19 68 75 107 53 72 94 152 30 45 123 135 37 58 80 67 9 22 61 78 81 119 38 43 48 76 124 35 65 104 121 66 47 16 93 74 82 8 165 144 49 86 113 149 12 26 41 92 103 150 87 2 28 89 105 128 132 143 23 64 91 11 118 126 159 46 116 130 97 101 114 117 134 25 55 170]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[worker_batch_size:500 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 replication_type:capi filter_expression: active:true checkpoint_interval:1800 target_nozzle_per_node:2 log_level:Info doc_batch_size_kb:2048 stats_interval:1000]
CheckpointManager 2017-04-25T13:54:57.645-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511...
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:54:57.652-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
StatisticsManager 2017-04-25T13:54:57.652-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:54:57.653-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:54:57.653-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:54:57.653-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:54:57.653-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:54:57.653-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:54:57.653-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:54:57.653-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:54:57.653-05:00 [INFO] Checkpointing starts in 1803.648 sec
PipelineRuntimeCtx 2017-04-25T13:54:57.653-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:54:57.653-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:54:57.655-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:54:57.655-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:54:57.657-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
ToplogyChangeDetector 2017-04-25T13:54:57.681-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:54:57.692-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:54:57.692-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:54:57.693-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:54:57.712-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:54:57.712-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CapiUtils 2017-04-25T13:54:57.715-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:54:57.715-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:54:57.715-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:54:57.722-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:54:57.731-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:54:57.735-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:54:57.735-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:57.735-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:54:57.735-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:54:57.735-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:54:57.735-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:54:57.755-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:54:57.755-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:54:57.755-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:54:57.768-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:54:57.768-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:54:57.820-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:54:57.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:54:57.899-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:54:57.899-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:54:57.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:54:57.899-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:54:57.899-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:54:57.900-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511 is started----------
PipelineManager 2017-04-25T13:54:57.900-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:54:57.900-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:54:57.900-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:54:58.480-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:54:58.481-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:54:58.481-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:54:58.481-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:54:58 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 100 successful
CapiNozzle 2017-04-25T13:54:58.646-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:54:58.667-05:00 [ERROR] Received malformed response from tcp connection vb=101. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f6/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}

CapiNozzle 2017-04-25T13:54:58.667-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err Received malformed response from tcp connection vb=101. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f6/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}
.
2017/04/25 13:54:58 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:54:58.703-05:00 [INFO] Send batch count=385
CapiNozzle 2017-04-25T13:54:58.713-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
2017/04/25 13:54:58 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 4 successful
StatisticsManager 2017-04-25T13:54:58.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:54:58 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:54:58.740-05:00 [INFO] Send batch count=193
CapiNozzle 2017-04-25T13:54:58.803-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:54:58.803-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:54:58 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:54:58.825-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:54:58 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:54:58 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 232 successful
CapiNozzle 2017-04-25T13:54:59.236-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:54:59 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 138 successful
CapiNozzle 2017-04-25T13:54:59.259-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:54:59 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:54:59.443-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:54:59 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 53 successful
CapiNozzle 2017-04-25T13:54:59.456-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:54:59 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 62 successful
CapiNozzle 2017-04-25T13:54:59.596-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:54:59 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 171 successful
CapiNozzle 2017-04-25T13:54:59.628-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:54:59 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 63 successful
StatisticsManager 2017-04-25T13:54:59.710-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:54:59 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 181 successful
CapiNozzle 2017-04-25T13:54:59.827-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:54:59.843-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
2017/04/25 13:54:59 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 94 successful
CheckpointManager 2017-04-25T13:54:59.902-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
CheckpointManager 2017-04-25T13:54:59.909-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:54:59 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:54:59 UPR_STREAMREQ for vb 95 successful
DcpNozzle 2017-04-25T13:55:00.018-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:55:00.038-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:55:00.090-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:00.109-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:55:00.498-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:55:00.498-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:55:00.498-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:55:00.498-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
ReplicationManager 2017-04-25T13:55:00.498-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
PipelineManager 2017-04-25T13:55:00.506-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:55:00.506-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:55:00.509-05:00 [INFO] err_list=[{"time":"2017-04-25T13:55:00.509292575-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:55:00.509-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:55:00.594-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:00.594-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:00.614-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
StatisticsManager 2017-04-25T13:55:00.655-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:55:00.655-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511 {"Errors": "[{\"time\":\"2017-04-25T13:55:00.509292575-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 0, "dcp_dispatch_time": 8, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 290, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 45158, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 290, "set_filtered": 0, "set_received_from_dcp": 45158, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:55:00.663-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:00.663-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:55:00.663-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:55:00.663-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:00.663-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:00.719-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:00.719-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:00.739-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:55:01.010-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:55:01.028-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:55:01.187-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:01.203-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:55:01.465-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:55:01.491-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:55:01.726-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:01.745-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:55:01.962-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:55:01.962-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:55:01.962-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:55:01.962-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:01.962-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:02.052-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:02.062-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:02.078-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:55:02.216-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:02.216-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:55:02.216-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:55:02.216-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:02.217-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:02.322-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:02.322-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:02.338-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:55:02.554-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:55:02.575-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:55:02.820-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:02.833-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:55:03.046-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:55:03.066-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:55:03.205-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:03.231-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:55:03.424-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:55:03.424-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:55:03.424-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:55:03.424-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:03.424-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:03.540-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:03.540-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:03.552-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:55:03.681-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:03.681-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:55:03.681-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:55:03.681-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:03.681-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:03.776-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:03.776-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:03.804-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:55:03.976-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:03.994-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
CapiNozzle 2017-04-25T13:55:04.205-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:04.232-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:55:04.435-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:04.464-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:55:04.614-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:04.634-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:55:04.786-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:04.786-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:55:04.786-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:55:04.786-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:04.787-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:04.873-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:04.873-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:04.895-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:55:05.009-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:05.009-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:55:05.009-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:55:05.009-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:05.009-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:05.120-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:05.120-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:05.140-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:55:05.297-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:05.316-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:55:05.524-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:05.602-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:55:05.734-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:05.747-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:55:05.970-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:05.984-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:55:06.320-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:06.320-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:55:06.320-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:55:06.320-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:06.320-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:06.441-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:06.441-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:06.452-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:55:06.619-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:06.619-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:55:06.619-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:55:06.619-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:06.619-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:06.717-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:06.717-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:06.728-05:00 [INFO] Retrying update docs for vb 104 for the 1th time
CapiNozzle 2017-04-25T13:55:07.056-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:55:07.066-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:55:07.329-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:55:07.340-05:00 [INFO] Retrying update docs for vb 104 for the 2th time
CapiNozzle 2017-04-25T13:55:07.528-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:55:07.539-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
GenericSupervisor 2017-04-25T13:55:07.655-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:55:07.655-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:55:07.655-05:00 [INFO] Finish executing 0x681fb0
ToplogyChangeDetector 2017-04-25T13:55:07.858-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:55:07.863-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:55:07.874-05:00 [INFO] Retrying update docs for vb 104 for the 3th time
CapiNozzle 2017-04-25T13:55:08.407-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:55:08.407-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:55:08.407-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:55:08.407-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:08.407-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:08.512-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:08.512-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:08.523-05:00 [INFO] Retrying update docs for vb 203 for the 1th time
CapiNozzle 2017-04-25T13:55:08.780-05:00 [ERROR] batchUpdateDocs for vb 104 failed with err receieved unexpected status code, 500, from update docs request for vb 104
.
CapiNozzle 2017-04-25T13:55:08.780-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 104 after 3 retries
CapiNozzle 2017-04-25T13:55:08.780-05:00 [ERROR] Raise error condition batch update docs failed for vb 104 after 3 retries
GenericSupervisor 2017-04-25T13:55:08.780-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:08.780-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:08.876-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:08.876-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:08.887-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:55:09.169-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:55:09.180-05:00 [INFO] Retrying update docs for vb 203 for the 2th time
CapiNozzle 2017-04-25T13:55:09.372-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:55:09.383-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:55:09.542-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:55:09.553-05:00 [INFO] Retrying update docs for vb 203 for the 3th time
CapiNozzle 2017-04-25T13:55:09.710-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:55:09.721-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:55:09.904-05:00 [ERROR] batchUpdateDocs for vb 203 failed with err receieved unexpected status code, 500, from update docs request for vb 203
.
CapiNozzle 2017-04-25T13:55:09.904-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 203 after 3 retries
CapiNozzle 2017-04-25T13:55:09.904-05:00 [ERROR] Raise error condition batch update docs failed for vb 203 after 3 retries
GenericSupervisor 2017-04-25T13:55:09.904-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:09.904-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:09.991-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:09.991-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:10.002-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:55:10.047-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:55:10.047-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:55:10.047-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:55:10.047-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:10.047-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:10.133-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:10.133-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:10.144-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
PipelineManager 2017-04-25T13:55:10.200-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:55:00.509292575-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]}
]
CapiNozzle 2017-04-25T13:55:10.401-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err receieved unexpected status code, 500, from update docs request for vb 301
.
CapiNozzle 2017-04-25T13:55:10.412-05:00 [INFO] Retrying update docs for vb 301 for the 2th time
PipelineManager 2017-04-25T13:55:10.509-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:55:10.509-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:55:10.509-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:55:10.509-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
PipelineRuntimeCtx 2017-04-25T13:55:10.509-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:55:10.509-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:10.509-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:55:10.510-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:10.509923321 -0500 CDT
GenericSupervisor 2017-04-25T13:55:10.510-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:10.510-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:55:10.510-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:55:10.510-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:55:10.510-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:55:10.510-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:55:10.510-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:55:10.510-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:55:10.510-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:55:10.510-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-612655511 has been stopped
AsyncListener 2017-04-25T13:55:10.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-612655511 is stopped
PipelineManager 2017-04-25T13:55:10.510-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-612655511
PipelineManager 2017-04-25T13:55:10.510-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:55:00.509292575-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

AsyncListener 2017-04-25T13:55:10.510-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:55:10.510-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
PipelineManager 2017-04-25T13:55:10.510-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:55:10.510-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:55:10.510-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.510-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:55:10.510-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:55:10.511-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
DcpNozzle 2017-04-25T13:55:10.511-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
DcpNozzle 2017-04-25T13:55:10.511-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
CapiNozzle 2017-04-25T13:55:10.511-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
CapiNozzle 2017-04-25T13:55:10.511-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:55:10.511-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:10.511-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:10.511071953 -0500 CDT
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:55:10.511-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:10.511-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:10.511378621 -0500 CDT
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:10.511-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:55:10.512-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:55:10.512-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:55:10.512-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:10.512-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:55:10.512-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:55:10.512-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:55:10.512-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:55:10.512-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:10.512-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
GenericPipeline 2017-04-25T13:55:10.512-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:10.512-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:55:10.514-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:55:10.514-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:10.514635643 -0500 CDT
DcpNozzle 2017-04-25T13:55:10.514-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:55:10.514-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:10.514-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:55:10.514-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:55:10.514-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:10.514943771 -0500 CDT
DcpNozzle 2017-04-25T13:55:10.515-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:55:10.515-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:10.515-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:55:10 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:55:10 Stream Closed for vb 172 StreamEnd simulated
2017/04/25 13:55:10 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:55:10.628-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:55:10.628-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:55:10.628-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:55:10.693-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:55:10.694-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:55:10.694-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:55:10.694-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:55:10.694-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:55:10.694-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:55:10.694-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:55:10.695-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:55:10.695-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
GenericSupervisor 2017-04-25T13:55:10.697-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:55:10.697-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
GenericSupervisor 2017-04-25T13:55:10.697-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:10.697-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:10.697-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:10.697-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:10.697-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:55:10.697-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
CheckpointManager 2017-04-25T13:55:10.697-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
CapiUtils 2017-04-25T13:55:10.774-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:55:10.775-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:55:10.775-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:55:10.775-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
StatisticsManager 2017-04-25T13:55:10.776-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:55:10.776-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
XDCRFactory 2017-04-25T13:55:10.776-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:55:10.776-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481 is constructed, start it
GenericPipeline 2017-04-25T13:55:10.777-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[31 115 155 25 96 98 134 148 156 13 21 116 158 49 106 110 10 5 46 68 94 111 169 170 112 36 85 92 131 147 165 11 75 93 83 157 65 123 133 47 100 124 118 152 168 70 1 29 32 35 44 54 56 79 139 51 60 87 119 86 120 128 145 41 78 80 95 33 50 61 109 18 42 81 84 132 140 161 8 99 7 20 55 67 102 105 16 9 74 39 143 48 64 40 153 163 88 154 15 149 126 129 136 19 76 90 159 27 30 34 37 58 114 24 69 121 162 167 43 63 82 107 117 146 89 164 4 6 22 52 113 122 150 166 72 97 108 144 77 138 160 12 66 91 135 71 142 3 53 45 101 23 57 151 2 38 59 103 137 141 73 0 62 14 17 26 104 130 28 125 127] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[179 242 275 281 338 339 224 305 231 319 293 211 214 251 280 178 197 215 258 273 289 300 309 260 302 269 325 188 209 174 314 315 229 248 249 250 304 331 303 324 172 238 217 219 262 270 203 233 313 201 307 333 226 225 276 190 191 196 216 230 235 186 202 278 316 200 267 292 301 204 291 187 183 279 295 318 206 227 246 265 298 189 237 255 297 323 335 198 234 322 240 243 171 185 259 261 264 182 239 263 266 274 299 257 290 199 205 221 294 312 252 332 222 286 296 306 285 321 337 195 212 277 287 317 330 210 283 284 208 193 213 218 271 329 340 173 177 180 228 268 308 175 176 181 207 232 282 327 245 311 236 254 256 336 244 288 334 194 223 184 220 241 247 310 253 192 272 320 326 328]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[293 211 214 251 280 178 309 197 215 258 273 289 300 260 302 269 325 188 209 174 314 315 250 304 229 248 249 331 303 324 172 238 217 219 262 270 203 233 313 201 307 333 226 225 276 196 190 191 216 230 235 278 186 202 316 200 267 292 301 204 291 187 183 279 295 318 206 227 246 265 298 323 335 189 237 255 297 198 234 322 240 243 261 264 171 185 259 274 182 239 263 266 299 257 290 199 205 221 294 312 252 332 222 286 296 306 285 321 337 330 195 212 277 287 317 210 283 284 208 271 329 193 213 218 340 180 228 173 177 268 308 232 282 175 176 181 207 327 245 311 336 236 254 256 244 288 334 194 223 184 220 241 247 310 253 192 328 272 320 326 179 242 275 281 338 339 305 224 231 319] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[49 106 110 10 170 5 46 68 94 111 169 112 36 85 92 131 147 165 11 75 93 83 157 65 123 133 47 100 124 118 152 168 70 56 79 1 29 32 35 44 54 139 51 60 87 119 86 120 128 145 41 78 80 95 33 50 61 109 161 18 42 81 84 132 140 8 99 7 20 55 67 102 105 16 9 74 39 143 48 64 40 153 163 88 154 15 149 126 129 136 19 76 90 159 27 30 34 37 58 114 24 69 121 162 167 43 63 82 107 117 146 89 164 150 4 6 22 52 113 122 166 72 97 108 144 77 138 160 12 66 91 135 71 142 3 53 45 101 23 57 151 2 38 59 103 137 141 73 0 62 14 17 26 104 130 28 125 127 31 115 155 25 96 98 134 148 156 13 21 116 158]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[worker_batch_size:500 optimistic_replication_threshold:256 source_nozzle_per_node:2 stats_interval:1000 filter_expression: checkpoint_interval:1800 doc_batch_size_kb:2048 failure_restart_interval:10 target_nozzle_per_node:2 log_level:Info replication_type:capi active:true]
CheckpointManager 2017-04-25T13:55:10.778-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481...
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:55:10.785-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
GenericSupervisor 2017-04-25T13:55:10.785-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:10.785-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:55:10.785-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:55:10.785-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:55:10.785-05:00 [INFO] Checkpointing starts in 1800.694 sec
PipelineRuntimeCtx 2017-04-25T13:55:10.785-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:55:10.785-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:55:10.786-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:55:10.786-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:55:10.787-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:55:10.787-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:55:10.787-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:55:10.787-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:55:10.791-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:55:12.771-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:55:12.771-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:55:12.771-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:55:12.792-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:55:12.807-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:55:12.835-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:55:12.848-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:55:12.915-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:55:12.915-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CheckpointManager 2017-04-25T13:55:12.918-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:55:12.919-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:55:12.932-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:55:12.932-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:55:13.022-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:55:13.022-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:55:13.022-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:55:13.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:55:13.043-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:55:13.044-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:55:13.141-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:55:13.141-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:55:13.141-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:55:13.162-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:55:13.162-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:55:13.326-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:55:13.326-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:55:13.326-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:55:13.362-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:55:13.362-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:55:13.362-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:55:13.362-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481 is started----------
PipelineManager 2017-04-25T13:55:13.363-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:55:13.363-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:55:13.363-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
StatisticsManager 2017-04-25T13:55:13.866-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CheckpointManager 2017-04-25T13:55:14.569-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:55:14.570-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:55:14.570-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:55:14.571-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:55:14 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 0 successful
CapiNozzle 2017-04-25T13:55:14.641-05:00 [INFO] Send batch count=224
2017/04/25 13:55:14 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 101 successful
CapiNozzle 2017-04-25T13:55:14.710-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:14.710-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:55:14 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 302 successful
CapiNozzle 2017-04-25T13:55:14.746-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:55:14 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 104 successful
CapiNozzle 2017-04-25T13:55:14.785-05:00 [INFO] Send batch count=66
2017/04/25 13:55:14 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 204 successful
CapiNozzle 2017-04-25T13:55:14.842-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:14.842-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:55:14.848-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:55:14.875-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
2017/04/25 13:55:14 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:55:14 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 117 successful
CapiNozzle 2017-04-25T13:55:15.082-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:55:15 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 220 successful
CapiNozzle 2017-04-25T13:55:15.121-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:55:15.141-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
2017/04/25 13:55:15 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 123 successful
CapiNozzle 2017-04-25T13:55:15.167-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
2017/04/25 13:55:15 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 230 successful
CapiNozzle 2017-04-25T13:55:15.333-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:15.354-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
2017/04/25 13:55:15 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 246 successful
CapiNozzle 2017-04-25T13:55:15.618-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:15.642-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:55:15 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 254 successful
CapiNozzle 2017-04-25T13:55:15.685-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:15.685-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:55:15.685-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:55:15.685-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]
ReplicationManager 2017-04-25T13:55:15.685-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]
PipelineManager 2017-04-25T13:55:15.695-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:55:15.695-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:55:15.698-05:00 [INFO] err_list=[{"time":"2017-04-25T13:55:15.698175188-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries"}]
PipelineManager 2017-04-25T13:55:15.698-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:55:15 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 261 successful
CapiNozzle 2017-04-25T13:55:15.791-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:15.791-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
StatisticsManager 2017-04-25T13:55:15.795-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:55:15.796-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481 {"Errors": "[{\"time\":\"2017-04-25T13:55:15.698175188-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 7, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 91, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 2801, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 91, "set_filtered": 0, "set_received_from_dcp": 2801, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:55:15.817-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:55:15 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:55:15 UPR_STREAMREQ for vb 273 successful
CapiNozzle 2017-04-25T13:55:16.028-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:16.028-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:55:16.028-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:55:16.028-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:16.028-05:00 [INFO] Send batch count=500
2017/04/25 13:55:16 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 91 successful
CapiNozzle 2017-04-25T13:55:16.157-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:16.158-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:55:16 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 88 successful
CapiNozzle 2017-04-25T13:55:16.182-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:55:16 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 192 successful
CheckpointManager 2017-04-25T13:55:16.304-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
CheckpointManager 2017-04-25T13:55:16.304-05:00 [INFO] Done with SetVBTimestamps
CapiNozzle 2017-04-25T13:55:16.304-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:55:16.329-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:55:16 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:55:16 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:55:16.459-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:55:16.465-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:55:16.725-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:16.745-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:55:16.897-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:55:16.913-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:55:17.102-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:17.118-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:55:17.290-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:55:17.290-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:55:17.290-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:55:17.290-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:17.290-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:17.403-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:17.403-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:17.429-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:55:17.432-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:17.432-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:55:17.432-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:55:17.432-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:17.432-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:17.526-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:17.526-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:17.538-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:55:17.819-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:17.848-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:55:18.106-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:18.123-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:55:18.271-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:18.289-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:55:18.457-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:18.483-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:55:18.779-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:18.779-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:55:18.779-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:55:18.779-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:18.779-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:18.855-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:18.855-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:18.869-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:55:18.915-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:18.915-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:55:18.915-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:55:18.915-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:18.915-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:18.979-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:18.980-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:19.016-05:00 [INFO] Retrying update docs for vb 102 for the 1th time
CapiNozzle 2017-04-25T13:55:19.256-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:55:19.271-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:55:19.436-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:55:19.457-05:00 [INFO] Retrying update docs for vb 102 for the 2th time
CapiNozzle 2017-04-25T13:55:19.678-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:55:19.699-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:55:19.904-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:55:19.924-05:00 [INFO] Retrying update docs for vb 102 for the 3th time
CapiNozzle 2017-04-25T13:55:20.151-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:55:20.151-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:55:20.151-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:55:20.151-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:20.151-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:20.215-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:20.215-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:20.230-05:00 [INFO] Retrying update docs for vb 202 for the 1th time
CapiNozzle 2017-04-25T13:55:20.379-05:00 [ERROR] batchUpdateDocs for vb 102 failed with err receieved unexpected status code, 500, from update docs request for vb 102
.
CapiNozzle 2017-04-25T13:55:20.379-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 102 after 3 retries
CapiNozzle 2017-04-25T13:55:20.379-05:00 [ERROR] Raise error condition batch update docs failed for vb 102 after 3 retries
GenericSupervisor 2017-04-25T13:55:20.379-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:20.379-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:20.459-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:20.459-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:20.496-05:00 [INFO] Retrying update docs for vb 2 for the 1th time
CapiNozzle 2017-04-25T13:55:20.514-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:20.533-05:00 [INFO] Retrying update docs for vb 202 for the 2th time
GenericSupervisor 2017-04-25T13:55:20.798-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:55:20.799-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:55:20.799-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:55:20.860-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:55:20.876-05:00 [INFO] Retrying update docs for vb 2 for the 2th time
CapiNozzle 2017-04-25T13:55:21.072-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:21.093-05:00 [INFO] Retrying update docs for vb 202 for the 3th time
CapiNozzle 2017-04-25T13:55:21.266-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:55:21.280-05:00 [INFO] Retrying update docs for vb 2 for the 3th time
CapiNozzle 2017-04-25T13:55:21.471-05:00 [ERROR] batchUpdateDocs for vb 202 failed with err receieved unexpected status code, 500, from update docs request for vb 202
.
CapiNozzle 2017-04-25T13:55:21.471-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 202 after 3 retries
CapiNozzle 2017-04-25T13:55:21.471-05:00 [ERROR] Raise error condition batch update docs failed for vb 202 after 3 retries
GenericSupervisor 2017-04-25T13:55:21.471-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:21.471-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:21.568-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:21.568-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:21.604-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:55:21.651-05:00 [ERROR] batchUpdateDocs for vb 2 failed with err receieved unexpected status code, 500, from update docs request for vb 2
.
CapiNozzle 2017-04-25T13:55:21.651-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 2 after 3 retries
CapiNozzle 2017-04-25T13:55:21.651-05:00 [ERROR] Raise error condition batch update docs failed for vb 2 after 3 retries
GenericSupervisor 2017-04-25T13:55:21.651-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:21.651-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:21.720-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:21.721-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:21.737-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:55:21.960-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:55:21.977-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:55:22.075-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:22.095-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:55:22.312-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:55:22.324-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:55:22.525-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:22.536-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
ToplogyChangeDetector 2017-04-25T13:55:22.953-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:55:23.157-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:55:23.158-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:55:23.158-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:55:23.158-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:23.158-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:23.274-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:23.275-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:23.290-05:00 [INFO] Retrying update docs for vb 303 for the 1th time
CapiNozzle 2017-04-25T13:55:23.727-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:23.727-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:55:23.727-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:55:23.727-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:23.728-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:23.820-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:23.821-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:23.831-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:55:24.454-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:55:24.465-05:00 [INFO] Retrying update docs for vb 303 for the 2th time
CapiNozzle 2017-04-25T13:55:24.663-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:24.674-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:55:24.985-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:55:24.995-05:00 [INFO] Retrying update docs for vb 303 for the 3th time
CapiNozzle 2017-04-25T13:55:25.196-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:25.207-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
PipelineManager 2017-04-25T13:55:25.209-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:55:15.698175188-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries]}
]
CapiNozzle 2017-04-25T13:55:25.430-05:00 [ERROR] batchUpdateDocs for vb 303 failed with err receieved unexpected status code, 500, from update docs request for vb 303
.
CapiNozzle 2017-04-25T13:55:25.430-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 303 after 3 retries
CapiNozzle 2017-04-25T13:55:25.430-05:00 [ERROR] Raise error condition batch update docs failed for vb 303 after 3 retries
GenericSupervisor 2017-04-25T13:55:25.430-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:25.430-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:25.520-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:25.520-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:25.532-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
CapiNozzle 2017-04-25T13:55:25.655-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:25.655-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:55:25.655-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:55:25.655-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:25.655-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:55:25.700-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries 
PipelineManager 2017-04-25T13:55:25.700-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:55:25.700-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:55:25.700-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
PipelineRuntimeCtx 2017-04-25T13:55:25.700-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:55:25.700-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:25.700-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:55:25.700-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:25.700439093 -0500 CDT
GenericSupervisor 2017-04-25T13:55:25.700-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:25.700-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:55:25.700-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:55:25.700-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:55:25.700-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:55:25.700-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:55:25.700-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:55:25.700-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:55:25.700-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:55:25.700-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-694861481 has been stopped
GenericPipeline 2017-04-25T13:55:25.700-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-694861481 is stopped
PipelineManager 2017-04-25T13:55:25.700-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:55:25.701-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:55:15.698175188-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 202 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:55:25.701-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:55:25.701-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
GenericPipeline 2017-04-25T13:55:25.701-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
GenericPipeline 2017-04-25T13:55:25.701-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
GenericPipeline 2017-04-25T13:55:25.701-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-694861481
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:25.701-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:25.702-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:25.702-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:25.702-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:25.702-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:25.702-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:55:25.702-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:25.702-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:55:25.702-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:55:25.702-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:55:25.702-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:55:25.702-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:55:25.702-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:25.702-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:55:25.702-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:55:25.702-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:55:25.702-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:55:25.702-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:55:25.702-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:25.703-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:25.703-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:25.702442374 -0500 CDT
DcpNozzle 2017-04-25T13:55:25.703-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:55:25.703-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:25.702808822 -0500 CDT
DcpNozzle 2017-04-25T13:55:25.703-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:55:25.703-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:55:25.704-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:55:25.704-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:55:25.704-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.704-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:55:25.704-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:25.704-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:55:25.707-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:55:25.707-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:55:25.707-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:25.707394121 -0500 CDT
DcpNozzle 2017-04-25T13:55:25.707-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:25.707414287 -0500 CDT
DcpNozzle 2017-04-25T13:55:25.707-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:55:25.707-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:55:25.707-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:25.707-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:55:25.707-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:25.707-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
AsyncListener 2017-04-25T13:55:25.707-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:25.707-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
2017/04/25 13:55:25 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:55:25 Stream Closed for vb 171 StreamEnd simulated
2017/04/25 13:55:25 Stream Closed for vb 172 StreamEnd simulated
CapiNozzle 2017-04-25T13:55:25.738-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:55:25.739-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:55:25.739-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:25.739-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:55:25.833-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:55:25.833-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:55:25.833-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:55:25.912-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:55:25.913-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:55:25.913-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:55:25.913-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:55:25.913-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:55:25.913-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:55:25.913-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:55:25.914-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:55:25.914-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
GenericSupervisor 2017-04-25T13:55:25.916-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:55:25.916-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
GenericSupervisor 2017-04-25T13:55:25.916-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:25.916-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:25.916-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:25.916-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:25.916-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:55:25.916-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
CheckpointManager 2017-04-25T13:55:25.916-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
CapiUtils 2017-04-25T13:55:25.946-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:55:25.946-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:55:25.946-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:55:25.946-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
StatisticsManager 2017-04-25T13:55:25.947-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:55:25.947-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
XDCRFactory 2017-04-25T13:55:25.947-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:55:25.947-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014 is constructed, start it
GenericPipeline 2017-04-25T13:55:25.948-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[267 274 326 327 173 272 335 221 224 311 216 338 208 199 258 187 330 336 200 206 210 260 310 319 322 325 307 195 237 309 339 196 287 186 209 227 251 283 340 324 299 174 245 262 275 240 301 333 278 243 201 181 171 194 280 176 215 242 255 304 234 214 189 265 286 178 217 303 331 188 244 306 297 277 261 239 281 285 175 248 225 268 291 183 246 226 263 308 191 231 320 295 192 193 233 266 273 337 250 282 198 249 254 257 298 329 184 321 317 332 211 229 334 264 271 312 288 290 305 323 256 276 296 313 220 289 259 315 328 197 203 182 314 190 228 300 185 238 252 293 318 219 222 253 207 179 235 241 316 247 284 177 218 230 213 223 269 292 294 172 204 205 212 202 270 279 302 180 232 236] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[129 154 24 54 134 82 95 4 16 58 67 164 38 41 103 19 112 25 34 114 84 110 1 11 111 75 81 87 118 48 8 109 55 107 113 148 26 57 88 157 137 30 47 155 158 61 73 9 70 5 39 143 149 153 29 45 78 131 18 33 97 6 23 37 126 27 117 168 105 166 94 96 130 160 170 119 127 63 77 80 115 125 2 35 123 146 13 92 72 124 144 98 106 59 141 65 69 62 99 139 152 156 163 52 53 116 159 93 140 17 60 89 167 31 50 42 169 0 14 49 121 10 142 132 15 90 20 22 145 101 133 147 162 7 56 40 3 100 21 83 76 71 108 138 36 46 68 79 12 51 120 122 128 64 102 135 151 28 43 150 86 91 44 85 161 165 32 136 104 66 74]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[40 3 100 21 83 76 36 46 71 108 138 12 51 68 79 64 102 120 122 128 28 43 135 151 86 91 150 44 85 32 136 161 165 66 74 104 129 24 54 154 134 4 16 82 95 58 67 38 41 164 103 19 25 34 112 114 1 11 84 110 75 81 111 87 118 48 8 109 55 107 26 57 113 148 88 157 30 47 137 61 73 155 158 9 5 39 70 29 45 143 149 153 18 33 78 131 6 23 97 37 27 117 126 105 166 168 94 96 119 127 130 160 170 2 35 63 77 80 115 125 13 92 123 146 72 98 106 124 144 59 65 69 141 163 52 53 62 99 139 152 156 116 159 93 17 60 140 31 50 89 167 0 14 42 169 49 121 10 142 132 15 90 20 22 145 7 56 101 133 147 162] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[314 185 190 228 300 238 252 293 318 207 219 222 253 241 179 235 247 284 316 177 218 230 269 213 223 292 294 172 204 205 212 302 202 270 279 180 232 236 173 267 274 326 327 272 335 216 221 224 311 338 208 187 199 258 200 330 336 319 206 210 260 310 322 325 195 307 237 309 339 196 287 340 186 209 227 251 283 299 324 174 245 262 275 240 301 333 278 201 243 181 171 194 280 255 176 215 242 304 214 234 189 265 286 331 178 217 303 188 244 306 277 297 261 239 281 285 175 248 225 268 291 183 246 191 226 263 308 231 295 320 273 192 193 233 266 337 250 282 329 184 198 249 254 257 298 321 317 332 211 229 334 264 271 312 288 290 305 323 220 256 276 296 313 289 197 259 315 328 203 182]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 log_level:Info active:true worker_batch_size:500 checkpoint_interval:1800 failure_restart_interval:10 stats_interval:1000 replication_type:capi filter_expression:]
CheckpointManager 2017-04-25T13:55:25.948-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014...
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:25.955-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
StatisticsManager 2017-04-25T13:55:25.955-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:55:25.961-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:55:25.961-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:55:25.961-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:55:25.961-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:25.961-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:55:25.961-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:55:25.961-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:55:25.961-05:00 [INFO] Checkpointing starts in 1804.371 sec
PipelineRuntimeCtx 2017-04-25T13:55:25.961-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:55:25.961-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:55:25.963-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:55:25.963-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:55:25.965-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:55:26.002-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:55:26.002-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:55:26.002-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:55:26.002-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:55:26.003-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:55:26.003-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:55:26.006-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:55:26.022-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:55:26.022-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:55:26.023-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:55:26.049-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:55:26.092-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:55:26.092-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:55:26.092-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:55:26.092-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:55:26.092-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:55:26.092-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:55:26.124-05:00 [INFO] Got the bucket metadata for ckmgr
CapiNozzle 2017-04-25T13:55:26.127-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:55:26.127-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:55:26.127-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:55:26.155-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:55:26.155-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
AsyncListener 2017-04-25T13:55:26.224-05:00 [INFO] Executing 0x793f70 timed out
AsyncListener 2017-04-25T13:55:26.224-05:00 [INFO] ****************************
AsyncListener 2017-04-25T13:55:26.224-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 failed to stop processing events. Leaving it alone to die. err=Execution timed out
DcpNozzle 2017-04-25T13:55:26.363-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:55:26.363-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:55:26.363-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
CheckpointManager 2017-04-25T13:55:26.372-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:55:26.372-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:55:26.372-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:55:26.373-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:55:26.384-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:55:26.384-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:55:26.384-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:55:26.385-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014 is started----------
PipelineManager 2017-04-25T13:55:26.385-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:55:26.385-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:55:26.385-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:55:26 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 1 successful
CapiNozzle 2017-04-25T13:55:26.441-05:00 [INFO] Send batch count=48
CapiNozzle 2017-04-25T13:55:26.451-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:26.451-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:26.471-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:55:26 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 200 successful
CapiNozzle 2017-04-25T13:55:26.549-05:00 [INFO] Send batch count=294
2017/04/25 13:55:26 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 15 successful
CapiNozzle 2017-04-25T13:55:26.632-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:26.679-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:55:26.692-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:26.692-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:55:26 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 215 successful
CapiNozzle 2017-04-25T13:55:26.719-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:55:26.787-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:55:26 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 230 successful
CapiNozzle 2017-04-25T13:55:26.805-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:55:26 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 132 successful
CapiNozzle 2017-04-25T13:55:26.935-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:26.935-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:55:26.935-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:55:26.935-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
ReplicationManager 2017-04-25T13:55:26.935-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]
PipelineManager 2017-04-25T13:55:26.945-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:55:26.945-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:55:26.950-05:00 [INFO] err_list=[{"time":"2017-04-25T13:55:26.950402638-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]
PipelineManager 2017-04-25T13:55:26.950-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
StatisticsManager 2017-04-25T13:55:26.972-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:55:26.972-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014 {"Errors": "[{\"time\":\"2017-04-25T13:55:26.950402638-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries]", "Status": "Pending"}

2017/04/25 13:55:26 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:55:26 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 250 successful
CapiNozzle 2017-04-25T13:55:27.094-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:27.101-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:55:27 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 148 successful
CapiNozzle 2017-04-25T13:55:27.137-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:55:27 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 257 successful
CapiNozzle 2017-04-25T13:55:27.201-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
2017/04/25 13:55:27 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 56 successful
CapiNozzle 2017-04-25T13:55:27.229-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
2017/04/25 13:55:27 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 170 successful
CapiNozzle 2017-04-25T13:55:27.581-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:55:27 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 188 successful
CapiNozzle 2017-04-25T13:55:27.604-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:55:27 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 97 successful
CheckpointManager 2017-04-25T13:55:27.752-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
CheckpointManager 2017-04-25T13:55:27.752-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:55:27 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:55:27 UPR_STREAMREQ for vb 196 successful
DcpNozzle 2017-04-25T13:55:27.797-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:55:27.886-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:55:27.897-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:55:27.911-05:00 [INFO] Retrying update docs for vb 306 for the 3th time
CapiNozzle 2017-04-25T13:55:28.144-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:28.160-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:55:28.310-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:55:28.310-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 306 after 3 retries
CapiNozzle 2017-04-25T13:55:28.310-05:00 [ERROR] Raise error condition batch update docs failed for vb 306 after 3 retries
GenericSupervisor 2017-04-25T13:55:28.310-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:28.310-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:28.395-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:28.395-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:28.412-05:00 [INFO] Retrying update docs for vb 306 for the 1th time
CapiNozzle 2017-04-25T13:55:28.528-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:55:28.528-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:55:28.528-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:55:28.528-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:28.528-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:28.694-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:28.694-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:28.725-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:55:28.778-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:55:28.809-05:00 [INFO] Retrying update docs for vb 306 for the 2th time
CapiNozzle 2017-04-25T13:55:29.264-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:29.280-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:55:29.863-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:55:29.888-05:00 [INFO] Retrying update docs for vb 306 for the 3th time
CapiNozzle 2017-04-25T13:55:30.048-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:30.071-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:55:30.240-05:00 [ERROR] batchUpdateDocs for vb 306 failed with err receieved unexpected status code, 500, from update docs request for vb 306
.
CapiNozzle 2017-04-25T13:55:30.240-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 306 after 3 retries
CapiNozzle 2017-04-25T13:55:30.240-05:00 [ERROR] Raise error condition batch update docs failed for vb 306 after 3 retries
GenericSupervisor 2017-04-25T13:55:30.240-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:30.240-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:30.355-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:30.355-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:30.367-05:00 [INFO] Retrying update docs for vb 302 for the 1th time
CapiNozzle 2017-04-25T13:55:30.511-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:55:30.511-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:55:30.511-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:55:30.511-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:30.511-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:30.704-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:30.704-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:30.745-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:30.746-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:55:30.762-05:00 [INFO] Retrying update docs for vb 302 for the 2th time
CapiNozzle 2017-04-25T13:55:31.186-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:31.214-05:00 [INFO] Retrying update docs for vb 302 for the 3th time
CapiNozzle 2017-04-25T13:55:31.499-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:31.514-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:55:31.561-05:00 [ERROR] batchUpdateDocs for vb 302 failed with err receieved unexpected status code, 500, from update docs request for vb 302
.
CapiNozzle 2017-04-25T13:55:31.561-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 302 after 3 retries
CapiNozzle 2017-04-25T13:55:31.561-05:00 [ERROR] Raise error condition batch update docs failed for vb 302 after 3 retries
GenericSupervisor 2017-04-25T13:55:31.561-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:31.561-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:31.649-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:31.649-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:31.662-05:00 [INFO] Retrying update docs for vb 206 for the 1th time
CapiNozzle 2017-04-25T13:55:32.937-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:55:32.941-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:32.960-05:00 [INFO] Retrying update docs for vb 206 for the 2th time
CapiNozzle 2017-04-25T13:55:32.963-05:00 [INFO] Retrying update docs for vb 1 for the 3th time
CapiNozzle 2017-04-25T13:55:35.587-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
CapiNozzle 2017-04-25T13:55:35.591-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:55:35.591-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 1 after 3 retries
CapiNozzle 2017-04-25T13:55:35.591-05:00 [ERROR] Raise error condition batch update docs failed for vb 1 after 3 retries
GenericSupervisor 2017-04-25T13:55:35.591-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:55:35.591-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:55:35.630-05:00 [INFO] Retrying update docs for vb 206 for the 3th time
CapiNozzle 2017-04-25T13:55:35.701-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:35.701-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:55:35.718-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
GenericSupervisor 2017-04-25T13:55:36.105-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:55:36.259-05:00 [INFO] Pipeline is no longer running, exit.
ToplogyChangeDetector 2017-04-25T13:55:36.354-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:55:36.373-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:55:36.644-05:00 [ERROR] batchUpdateDocs for vb 206 failed with err receieved unexpected status code, 500, from update docs request for vb 206
.
PipelineManager 2017-04-25T13:55:38.542-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries 
PipelineManager 2017-04-25T13:55:43.109-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:55:43.189-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:55:43.193-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
PipelineRuntimeCtx 2017-04-25T13:55:43.197-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:55:43.201-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:55:43.235-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 206 after 3 retries
CapiNozzle 2017-04-25T13:55:43.235-05:00 [ERROR] Raise error condition batch update docs failed for vb 206 after 3 retries
GenericSupervisor 2017-04-25T13:55:43.218-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:55:43.252-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:43.203035477 -0500 CDT
GenericSupervisor 2017-04-25T13:55:43.242-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 206 after 3 retries], but error is ignored. pipeline_state=3
GenericSupervisor 2017-04-25T13:55:43.392-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:43.392-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:55:43.410-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:55:43.416-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:55:43.427-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:55:43.427-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CapiNozzle 2017-04-25T13:55:43.447-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:55:43.474-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:55:43.499-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:55:43.499-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:55:43.500-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:55:43.500-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-913897014 has been stopped
GenericPipeline 2017-04-25T13:55:43.648-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-913897014 is stopped
PipelineManager 2017-04-25T13:55:43.648-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:55:43.648-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:55:26.950402638-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:55:43.685-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:55:43.706-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
GenericPipeline 2017-04-25T13:55:43.710-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
GenericPipeline 2017-04-25T13:55:43.710-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
GenericPipeline 2017-04-25T13:55:43.710-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-913897014
CapiNozzle 2017-04-25T13:55:43.719-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
AsyncListener 2017-04-25T13:55:43.711-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:43.723-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:43.724-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:55:43.724-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:55:43.724-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:55:43.725-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:43.725-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:55:43.731-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:55:43.731-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:55:43.733-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:55:43.736-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
DcpNozzle 2017-04-25T13:55:43.736-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:43.747-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:55:43.747-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:55:43.747-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:55:43.747-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:55:43.747-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:55:43.748-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:43.731809705 -0500 CDT
CapiNozzle 2017-04-25T13:55:43.773-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:43.747364742 -0500 CDT
DcpNozzle 2017-04-25T13:55:43.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:55:43.773-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:55:43.773-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:55:43.793-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:43.797-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:55:43.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
2017/04/25 13:55:43 Stream Closed for vb 0 StreamEnd simulated
DcpNozzle 2017-04-25T13:55:43.900-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:55:43.900-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:43.900458697 -0500 CDT
DcpNozzle 2017-04-25T13:55:43.900-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:55:43.900-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:43.907-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:55:43.923-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:55:43.951-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:55:43.951-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:43.951-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:55:43.958-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:55:43.958-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:55:43.958815123 -0500 CDT
DcpNozzle 2017-04-25T13:55:43.958-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:55:43.958-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:43.958-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
CapiNozzle 2017-04-25T13:55:43.959-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:55:43.959-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:55:43.960-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:55:43.960-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
PipelineManager 2017-04-25T13:55:44.062-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:55:26.950402638-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:55:44.239-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:55:44.243-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:55:44.243-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
ReplicationManager 2017-04-25T13:55:44.268-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
XDCRFactory 2017-04-25T13:55:44.384-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:55:44.387-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:55:44.396-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:55:44.396-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:55:44.396-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:55:44.396-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:55:44.396-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:55:44.398-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:55:44.402-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
GenericSupervisor 2017-04-25T13:55:44.407-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:55:44.407-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
GenericSupervisor 2017-04-25T13:55:44.407-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:44.408-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:44.408-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:44.408-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:55:44.408-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:55:44.424-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
CheckpointManager 2017-04-25T13:55:44.424-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
CapiUtils 2017-04-25T13:55:44.480-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:55:44.481-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:55:44.482-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:55:44.485-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
StatisticsManager 2017-04-25T13:55:44.493-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:55:44.494-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
XDCRFactory 2017-04-25T13:55:44.494-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:55:44.494-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 is constructed, start it
GenericPipeline 2017-04-25T13:55:44.496-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[111 170 148 132 145 159 166 4 73 127 128 112 169 88 92 0 9 23 26 60 76 46 78 113 151 7 138 143 40 72 146 157 109 124 165 49 57 87 96 29 52 99 163 117 140 81 10 118 160 108 44 168 120 155 58 110 154 55 56 83 89 32 105 126 141 152 15 31 123 79 137 5 13 41 106 115 19 45 59 85 27 54 100 134 17 24 65 22 39 80 91 16 62 103 114 6 50 119 153 43 98 67 14 161 116 30 36 71 131 75 12 33 61 97 28 20 37 53 125 51 150 18 38 3 102 122 8 34 69 82 147 135 139 167 2 66 90 104 25 47 86 121 1 95 107 149 21 136 64 93 156 42 63 74 144 94 133 35 101 129 164 11 130 142 158 162 48 68 77 84 70] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[243 314 179 228 299 190 211 239 295 210 302 200 204 308 313 327 247 284 270 180 217 230 325 331 337 176 186 265 281 278 298 262 246 258 272 340 332 209 229 252 215 242 285 316 208 324 338 312 253 304 287 289 334 213 221 283 315 199 202 203 224 227 263 178 184 291 321 256 282 245 249 286 317 319 191 181 267 280 175 187 197 232 329 171 318 328 255 306 259 182 192 275 322 294 173 290 218 311 339 185 195 297 194 222 301 310 174 257 296 193 241 323 326 279 172 236 276 266 277 330 307 223 231 250 261 269 336 214 216 235 188 207 260 273 225 189 251 212 234 201 335 320 206 183 198 205 237 238 233 305 292 333 248 300 219 303 244 268 288 293 196 226 254 309 264 220 240 177 271 274]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[209 229 252 332 208 215 242 285 316 324 338 253 304 312 287 289 334 199 213 221 283 315 202 203 178 184 224 227 263 256 282 291 321 245 249 286 317 319 191 181 267 280 175 187 197 232 329 171 318 255 306 328 182 192 259 275 322 294 173 290 218 311 339 185 195 297 194 222 301 310 174 257 296 193 241 323 326 172 236 276 279 266 277 330 307 223 231 250 261 214 216 235 269 336 188 207 260 273 225 189 251 201 212 234 320 335 206 183 198 205 237 238 233 305 292 333 219 248 300 244 268 303 288 293 196 226 254 309 220 240 264 177 271 274 243 314 179 228 190 211 239 295 299 210 200 204 302 247 284 308 313 327 270 180 217 230 325 176 186 265 281 331 337 262 278 298 246 258 272 340] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[29 52 99 163 117 140 81 10 118 160 108 44 168 120 155 58 55 56 83 89 110 154 32 105 126 141 152 15 31 123 79 137 5 13 41 19 45 59 85 106 115 27 54 100 134 17 24 65 22 39 80 91 16 62 103 114 6 50 119 153 43 98 67 14 161 116 30 36 71 131 75 12 33 61 97 28 20 37 53 125 51 150 18 38 3 102 122 8 34 69 82 147 2 66 90 104 135 139 167 25 47 86 121 1 95 107 149 21 136 64 93 42 63 74 144 156 94 133 35 101 129 164 11 130 48 68 77 84 142 158 162 70 111 170 148 4 73 127 128 132 145 159 166 112 169 88 92 0 9 23 26 60 76 46 78 113 151 7 138 143 40 72 146 157 49 57 87 96 109 124 165]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi checkpoint_interval:1800 worker_batch_size:500 log_level:Info filter_expression: active:true]
CheckpointManager 2017-04-25T13:55:44.499-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420...
AsyncListener 2017-04-25T13:55:44.540-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:55:44.541-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
CheckpointManager 2017-04-25T13:55:44.555-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:55:44.558-05:00 [INFO] Checkpointing starts in 1803.711 sec
PipelineRuntimeCtx 2017-04-25T13:55:44.558-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:55:44.558-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:55:44.558-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:55:44.560-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:55:44.562-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:55:44.562-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:55:44.562-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:55:44.563-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:55:44.563-05:00 [INFO] Service PipelineSupervisor has been started
StatisticsManager 2017-04-25T13:55:44.564-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:55:44.578-05:00 [INFO] monitorPipelineHealth started
CapiNozzle 2017-04-25T13:55:44.583-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:55:44.619-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:55:44.619-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:55:44.620-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
CapiUtils 2017-04-25T13:55:44.632-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:55:44.632-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
DcpNozzle 2017-04-25T13:55:44.632-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:55:44.632-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:55:44.633-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:55:44.678-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:55:44.695-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:55:44.705-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:55:44.802-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:55:44.802-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:55:44.802-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:55:44.808-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:55:44.809-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:55:44.809-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:55:44.871-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:55:44.871-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:55:44.871-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:55:44.892-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:55:44.892-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:55:44.923-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:55:45.136-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:55:45.136-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:55:45.136-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:55:45.161-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:55:45.161-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:55:45.161-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:55:45.171-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 is started----------
PipelineManager 2017-04-25T13:55:45.173-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:55:45.174-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:55:45.174-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:55:45.282-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:55:45.296-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:55:45.296-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:55:45.298-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:55:45 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:55:45 UPR_STREAMREQ for vb 105 successful
CapiNozzle 2017-04-25T13:55:46.359-05:00 [INFO] Send batch count=249
GenericSupervisor 2017-04-25T13:55:47.486-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
CapiNozzle 2017-04-25T13:55:48.095-05:00 [INFO] Send batch count=338
GenericSupervisor 2017-04-25T13:55:48.253-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
2017/04/25 13:55:50 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:55:52 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:55:52 UPR_STREAMREQ for vb 106 successful
StatisticsManager 2017-04-25T13:55:53.002-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:55:53 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 206 successful
CapiNozzle 2017-04-25T13:55:53.412-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:55:53.427-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:55:53 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 307 successful
CapiNozzle 2017-04-25T13:55:53.629-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:55:53 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:55:53 UPR_STREAMREQ for vb 8 successful
GenericSupervisor 2017-04-25T13:55:54.868-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:55:55.321-05:00 [INFO] validateSourceTopology...
2017/04/25 13:55:55 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:55:55 UPR_STREAMREQ for vb 111 successful
StatisticsManager 2017-04-25T13:55:55.706-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:55:55.714-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
GenericSupervisor 2017-04-25T13:55:55.767-05:00 [INFO] Executing 0x681fb0 timed out
2017/04/25 13:55:56 UPR_STREAMREQ for vb 308 successful
GenericSupervisor 2017-04-25T13:55:56.835-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:55:56.996-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:55:57.027-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
StatisticsManager 2017-04-25T13:55:57.092-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:55:26.950402638-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 741, "dcp_dispatch_time": 70, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 286, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 13361, "docs_rep_queue": 5954, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 11170, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 286, "set_filtered": 0, "set_received_from_dcp": 13361, "size_rep_queue": 7910387, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 4737}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 176, "docs_opt_repd": 0, "docs_rep_queue": 10921, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 176, "size_rep_queue": 13157141, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 277, "max": 4654, "mean": 4651.613718411552, "min": 4645}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 110, "docs_opt_repd": 0, "docs_rep_queue": 10676, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 110, "size_rep_queue": 13919644, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 210, "max": 4854, "mean": 4851.419047619048, "min": 4850}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 6607, "max": 124252, "mean": 134.224, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 12145, "expiry_received_from_dcp": 0, "set_received_from_dcp": 12145}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 6804, "max": 83, "mean": 9.43, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 11376, "expiry_received_from_dcp": 0, "set_received_from_dcp": 11376}}

StatisticsManager 2017-04-25T13:55:57.095-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 12145 items, sent 338 items
StatisticsManager 2017-04-25T13:55:57.095-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 11376 items, sent 249 items
StatisticsManager 2017-04-25T13:55:57.097-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:12145]
StatisticsManager 2017-04-25T13:55:57.099-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 12145 items, sent 12145 items. streams inactive: [31 45 69 48 65 73 13 46 77 127 168 113 116 145 21 61 72 80 112 159 163 39 126 148 170 57 86 123 134 124 136 147 24 59 78 97 121 164 133 32 58 128 132 20 30 51 68 95 119 157 34 49 90 162 161 27 38 53 89 143 12 52 99 115 120 142 36 87 88 37 75 165 76 93 35 43 54 169 135 41 85 125 98 141 25 44 60 64 74 144 166 155 14 66 139 150 79 15 17 28 29 50 40 42 138 16 22 146 70 122 129 154 140 152 56 62 63 83 91 151 11 19 33 47 158 10 26 71 81 156 23 160 18 67 82 118 55 92 94 137 84 114 131 149 153 96 130 117 167]
StatisticsManager 2017-04-25T13:55:57.099-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:11376]
StatisticsManager 2017-04-25T13:55:57.099-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 11376 items, sent 11376 items. streams inactive: [184 330 197 212 221 246 297 238 259 292 172 232 268 291 179 185 296 325 257 290 327 329 331 183 250 254 284 233 242 287 332 191 195 214 234 256 309 318 199 218 276 315 192 260 312 326 336 262 180 186 210 222 230 252 274 285 317 316 227 235 237 243 245 273 337 187 267 286 319 313 171 182 215 219 240 263 271 338 239 266 288 335 193 258 175 217 275 340 216 224 229 281 311 314 277 194 249 280 178 213 247 289 324 282 339 177 189 265 283 176 198 211 225 269 188 220 223 264 320 322 196 244 270 295 298 321 294 181 228 241 248 272 278 279 299 293 173 174 190 236 253 261 323 334 231 328 333 226 251 255 310]
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:55:57.100-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
2017/04/25 13:55:57 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 315 successful
StatisticsManager 2017-04-25T13:55:57.515-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:55:57 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 23 successful
StatisticsManager 2017-04-25T13:55:57.639-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:55:57 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 131 successful
CapiNozzle 2017-04-25T13:55:57.828-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:55:57 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 238 successful
CapiNozzle 2017-04-25T13:55:57.887-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:55:57.925-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:55:57 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:55:57 UPR_STREAMREQ for vb 42 successful
CapiNozzle 2017-04-25T13:55:57.961-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:55:58 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:55:58 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:55:59 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:55:59 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:55:59 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:55:59 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:55:59 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:55:59 UPR_STREAMREQ for vb 247 successful
CapiNozzle 2017-04-25T13:55:59.804-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
PipelineManager 2017-04-25T13:55:59.555-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:56:01 UPR_STREAMREQ for vb 150 successful
CapiNozzle 2017-04-25T13:56:02.982-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:56:01.859-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
GenericSupervisor 2017-04-25T13:56:04.276-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:56:04.702-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:56:04.881-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:56:04.740-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:56:04 UPR_STREAMREQ for vb 50 successful
GenericSupervisor 2017-04-25T13:56:05.014-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:56:05 UPR_STREAMREQ for vb 248 successful
DcpNozzle 2017-04-25T13:56:05.150-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
StatisticsManager 2017-04-25T13:56:05.243-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
ToplogyChangeDetector 2017-04-25T13:56:05.259-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:56:05.292-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
StatisticsManager 2017-04-25T13:56:05.465-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:56:05.471-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:55:26.950402638-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 0 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 131, "dcp_dispatch_time": 106, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 286, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 58885, "docs_rep_queue": 52717, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 7166, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 286, "set_filtered": 0, "set_received_from_dcp": 58885, "size_rep_queue": 67682314, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 4737}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 176, "docs_opt_repd": 0, "docs_rep_queue": 27357, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 176, "size_rep_queue": 35210424, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 277, "max": 4654, "mean": 4651.613718411552, "min": 4645}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 110, "docs_opt_repd": 0, "docs_rep_queue": 25360, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 110, "size_rep_queue": 32471890, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 210, "max": 4854, "mean": 4851.419047619048, "min": 4850}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 131, "dcp_dispatch_time": {"count": 30993, "max": 1353, "mean": 14.155, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 32392, "expiry_received_from_dcp": 0, "set_received_from_dcp": 32392}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 27892, "max": 195403, "mean": 208.08, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 29428, "expiry_received_from_dcp": 0, "set_received_from_dcp": 29428}}

StatisticsManager 2017-04-25T13:56:05.472-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 32719 items, sent 338 items
StatisticsManager 2017-04-25T13:56:05.472-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 29428 items, sent 249 items
StatisticsManager 2017-04-25T13:56:05.484-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:32719]
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 32719 items, sent 32719 items. streams inactive: [61 72 80 159 163 170 57 86 59 78 97 164 58 95 51 68 157 90 162 161 53 89 52 99 88 87 75 165 54 76 93 169 85 74 98 60 64 166 155 66 79 154 70 91 152 56 62 63 83 151 156 158 71 81 160 67 82 55 92 94 84 153 96 167 69 65 73 168 77]
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:29428]
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 29428 items, sent 29428 items. streams inactive: [199 276 192 260 252 262 180 186 274 285 273 187 267 286 263 271 171 182 266 288 193 258 175 275 281 277 194 249 280 178 289 282 177 189 265 283 269 176 198 188 264 298 196 270 295 272 278 279 294 181 299 253 261 293 173 174 190 251 255 184 297 197 259 292 172 268 291 179 185 296 257 290 284 183 250 254 287 256 191 195]
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =327 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =327 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:05.485-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
2017/04/25 13:56:05 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 53 successful
StatisticsManager 2017-04-25T13:56:05.643-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
2017/04/25 13:56:05 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:56:05 UPR_STREAMREQ for vb 55 successful
CapiNozzle 2017-04-25T13:56:06.905-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:56:06 UPR_STREAMREQ for vb 163 successful
CapiNozzle 2017-04-25T13:56:07.024-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:56:07.024-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:56:06.386-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:56:07.059-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:56:07.061-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:56:07 UPR_STREAMREQ for vb 164 successful
GenericSupervisor 2017-04-25T13:56:07.025-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
2017/04/25 13:56:07 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:56:06 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:56:07 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:56:08 UPR_STREAMREQ for vb 165 successful
CapiNozzle 2017-04-25T13:56:07.202-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
ReplicationManager 2017-04-25T13:56:08.503-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]
2017/04/25 13:56:08 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:56:08 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:56:08 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:56:09 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:56:09 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:56:09 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:56:09 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:56:09 UPR_STREAMREQ for vb 166 successful
CapiNozzle 2017-04-25T13:56:09.037-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:56:09.895-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
2017/04/25 13:56:10 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:56:10 UPR_STREAMREQ for vb 167 successful
GenericSupervisor 2017-04-25T13:56:10.277-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:56:10.477-05:00 [INFO] Send batch count=500
2017/04/25 13:56:10 UPR_STREAMREQ for vb 61 successful
StatisticsManager 2017-04-25T13:56:12.636-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
PipelineManager 2017-04-25T13:56:12.932-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
StatisticsManager 2017-04-25T13:56:12.933-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:56:13.004-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 0, "dcp_datach_length": 639, "dcp_dispatch_time": 618, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 286, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 1, "docs_received_from_dcp": 75927, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 286, "set_filtered": 0, "set_received_from_dcp": 75927, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

2017/04/25 13:56:13 UPR_STREAMREQ for vb 62 successful
PipelineManager 2017-04-25T13:56:12.976-05:00 [INFO] err_list=[{"time":"2017-04-25T13:56:12.976159412-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]
PipelineManager 2017-04-25T13:56:13.113-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:56:13.128-05:00 [INFO] Send batch count=500
2017/04/25 13:56:13 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 168 successful
DcpNozzle 2017-04-25T13:56:13.282-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 incrementing counter for inactive streams 1
2017/04/25 13:56:13 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 171 successful
CapiNozzle 2017-04-25T13:56:13.338-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:13.388-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:56:13.400-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:13.400-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:56:13.415-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:56:13.425-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:56:13 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 68 successful
PipelineManager 2017-04-25T13:56:13.471-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:56:12.976159412-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries]}
]
2017/04/25 13:56:13 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:56:13 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:56:14 UPR_STREAMREQ for vb 78 successful
GenericSupervisor 2017-04-25T13:56:14.625-05:00 [INFO] About to call function
2017/04/25 13:56:14 UPR_STREAMREQ for vb 184 successful
GenericSupervisor 2017-04-25T13:56:14.634-05:00 [INFO] Pipeline is no longer running, exit.
ToplogyChangeDetector 2017-04-25T13:56:15.368-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:56:15.457-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:56:15.583-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
GenericSupervisor 2017-04-25T13:56:15.155-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:56:16.009-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:56:16.138-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:56:16 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:56:16 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 93 successful
CheckpointManager 2017-04-25T13:56:17.177-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
CheckpointManager 2017-04-25T13:56:17.177-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:56:17 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:56:17 UPR_STREAMREQ for vb 99 successful
DcpNozzle 2017-04-25T13:56:17.338-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:56:17.278-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:56:17.493-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:56:17.577-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:56:17.678-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:56:17.705-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:56:19.591-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:56:19.709-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:56:19.709-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:56:19.735-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:56:19.735-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:56:19.735-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:56:19.735-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
GenericSupervisor 2017-04-25T13:56:19.761-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:56:19.779-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:56:19.799-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:56:20.042-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:20.052-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:56:20.032-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:20.059-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:56:20.104-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
CapiNozzle 2017-04-25T13:56:20.129-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:56:20.523-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:20.557-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:56:20.731-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:56:20.752-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
CapiNozzle 2017-04-25T13:56:21.099-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:21.111-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:56:21.313-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:56:21.326-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
CapiNozzle 2017-04-25T13:56:21.642-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:21.642-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:56:21.642-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:56:21.642-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:56:21.642-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:56:21.765-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:56:21.765-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:56:21.765-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:56:21.765-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:56:21.765-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:56:21.748-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:21.772-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:56:21.799-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
PipelineManager 2017-04-25T13:56:23.864-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries 
PipelineManager 2017-04-25T13:56:24.640-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:56:25.508-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:56:25.659-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
PipelineRuntimeCtx 2017-04-25T13:56:25.806-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:56:25.811-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
CapiNozzle 2017-04-25T13:56:25.824-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
GenericSupervisor 2017-04-25T13:56:25.855-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:56:25.873-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:25.813197021 -0500 CDT
GenericSupervisor 2017-04-25T13:56:25.887-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:56:25.887-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:56:25.932-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:56:25.933-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:56:25.934-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:56:25.934-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:56:25.936-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:56:25.937-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:56:25.938-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:56:25.938-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 has been stopped
GenericPipeline 2017-04-25T13:56:25.950-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-396882420 is stopped
PipelineManager 2017-04-25T13:56:25.950-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
GenericPipeline 2017-04-25T13:56:25.961-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
GenericPipeline 2017-04-25T13:56:25.965-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
GenericPipeline 2017-04-25T13:56:25.965-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
GenericPipeline 2017-04-25T13:56:25.965-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-396882420
AsyncListener 2017-04-25T13:56:25.966-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:25.971-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:25.973-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:25.973-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:25.973-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:25.973-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:56:25.974-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:56:25.974-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:56:25.980-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:56:25.980-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:56:25.981-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:56:25.983-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:56:26.000-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:56:26.000-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:56:26.000-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:56:26.000-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:26.000-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:56:26.000-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:25.980765342 -0500 CDT
DcpNozzle 2017-04-25T13:56:26.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:56:26.001-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:26.000430606 -0500 CDT
DcpNozzle 2017-04-25T13:56:26.001-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:56:26.001-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:56:26.005-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
PipelineManager 2017-04-25T13:56:26.013-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:56:12.976159412-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:56:26.017-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
AsyncListener 2017-04-25T13:56:26.016-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.020-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:26.020-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:26.021-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:56:26.030-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:56:26.033-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:56:26.033-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
GenericPipeline 2017-04-25T13:56:26.056-05:00 [INFO] Finish executing 0x65e930
CapiNozzle 2017-04-25T13:56:25.999-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
GenericPipeline 2017-04-25T13:56:26.120-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
CapiNozzle 2017-04-25T13:56:26.148-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:56:26.151-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:56:26.151-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:26.151-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:56:26.151-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:56:26.151-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:26.151783356 -0500 CDT
DcpNozzle 2017-04-25T13:56:26.151-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:56:26.152-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:26.152-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:56:26.155-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:56:26.155-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:26.155920938 -0500 CDT
DcpNozzle 2017-04-25T13:56:26.156-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:56:26.156-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:26.156-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
PipelineManager 2017-04-25T13:56:26.228-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:56:12.976159412-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 200 after 3 retries"}]}, progress={Start pipeline construction}
]
XDCRFactory 2017-04-25T13:56:26.316-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:56:26.324-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:56:26.324-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:56:26.413-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:56:26.417-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:56:26.419-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:56:26.420-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:56:26.420-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:56:26.420-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:56:26.420-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:56:26.423-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:56:26.427-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
GenericSupervisor 2017-04-25T13:56:26.438-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:56:26.438-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
GenericSupervisor 2017-04-25T13:56:26.441-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:26.441-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:26.441-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:26.441-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:26.441-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:56:26.445-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
CheckpointManager 2017-04-25T13:56:26.445-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
CapiUtils 2017-04-25T13:56:26.480-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:56:26.480-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:56:26.483-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:56:26.488-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
StatisticsManager 2017-04-25T13:56:26.498-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:56:26.501-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
XDCRFactory 2017-04-25T13:56:26.501-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:56:26.504-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 is constructed, start it
GenericPipeline 2017-04-25T13:56:26.506-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[339 252 284 194 323 247 265 230 310 215 242 302 321 179 245 281 326 220 285 328 218 300 303 173 214 234 180 260 318 333 228 237 175 213 206 263 253 246 250 211 241 267 208 216 273 298 271 272 311 181 297 210 248 295 338 324 207 340 209 322 188 232 278 288 313 244 274 176 330 337 184 251 336 226 258 200 327 261 316 335 185 190 270 331 243 238 240 287 199 203 275 294 177 221 236 289 229 225 172 262 264 174 202 227 257 308 320 259 280 256 307 178 198 255 283 319 186 187 171 293 304 183 205 223 249 269 325 329 306 201 222 235 277 314 305 196 224 231 286 197 315 217 254 291 301 239 219 233 192 212 182 268 282 317 195 290 292 332 299 189 266 279 296 309 312 191 193 204 276 334] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[7 26 55 158 100 124 16 74 84 133 159 22 25 29 53 94 148 44 120 139 143 160 98 145 42 144 4 36 129 33 114 30 93 162 5 27 151 14 34 87 121 128 57 65 79 82 67 56 101 117 122 3 68 71 81 106 167 48 90 97 132 146 59 95 134 19 51 99 165 170 70 118 153 166 38 49 80 137 15 138 46 66 58 123 126 141 8 50 163 18 164 40 43 69 155 169 20 41 91 92 102 112 12 73 24 61 108 77 85 168 28 125 127 1 116 130 152 154 62 149 140 109 147 13 45 64 75 76 104 157 11 103 83 9 89 37 131 150 31 86 107 110 111 113 2 60 142 6 35 17 21 135 10 32 54 88 96 0 23 63 105 161 47 52 78 39 72 136 156 115 119]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[218 300 303 173 214 234 180 260 318 333 228 237 175 213 206 253 263 246 250 211 241 267 208 216 273 298 271 272 311 181 297 210 248 295 338 207 324 209 340 188 232 278 322 288 313 176 244 274 330 337 184 251 226 258 336 200 327 261 316 335 185 190 270 331 238 240 243 199 203 275 287 177 221 236 289 294 225 229 172 174 202 227 257 262 264 308 320 259 280 178 198 255 256 307 283 319 171 186 187 293 304 183 205 223 249 269 325 329 306 201 222 235 277 314 196 224 231 286 305 197 315 217 254 291 301 239 219 233 182 192 212 268 282 317 195 290 292 332 189 266 279 296 299 191 193 204 276 309 312 334 252 284 339 194 323 247 265 230 310 215 242 302 321 179 245 281 326 220 285 328] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[42 144 4 36 129 33 114 30 93 162 5 27 151 14 34 87 121 128 57 65 79 82 67 56 101 117 122 3 68 71 81 106 167 48 90 97 132 146 59 95 134 19 51 99 165 170 70 118 153 166 38 49 80 137 15 138 46 66 58 123 126 141 8 50 163 18 164 40 43 69 155 169 20 41 91 92 102 112 12 73 24 61 108 77 85 168 28 125 127 1 116 130 152 154 62 149 140 13 45 64 75 76 104 109 147 157 11 103 83 9 89 37 131 150 31 86 107 110 111 113 2 60 142 6 35 17 21 135 10 32 54 88 96 0 23 63 105 161 47 52 78 39 72 136 156 115 119 7 26 55 158 100 124 16 74 84 133 22 25 29 53 94 148 159 44 120 139 143 160 98 145]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[checkpoint_interval:1800 source_nozzle_per_node:2 log_level:Info replication_type:capi filter_expression: doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 target_nozzle_per_node:2 stats_interval:1000 active:true worker_batch_size:500]
CheckpointManager 2017-04-25T13:56:26.509-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248...
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:56:26.535-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:56:26.536-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:56:26.537-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:56:26.537-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:56:26.539-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:56:26.538-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:56:26.541-05:00 [INFO] Checkpointing starts in 1803.584 sec
PipelineRuntimeCtx 2017-04-25T13:56:26.541-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:56:26.541-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:56:26.542-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:56:26.553-05:00 [INFO] Service StatisticsManager has been started
CapiUtils 2017-04-25T13:56:26.553-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:56:26.553-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:56:26.554-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:56:26.554-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:56:26.554-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:56:26.555-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:56:26.566-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:56:26.594-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:56:26.595-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:56:26.595-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:56:26.619-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:56:26.619-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:56:26.642-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:56:26.646-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:56:26.676-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:56:26.715-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:56:26.715-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:56:26.715-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:56:26.723-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:56:26.724-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:56:26.725-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:56:26.752-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:56:26.752-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:56:26.752-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:56:26.767-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:56:26.767-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:56:26.771-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:56:26.941-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:56:26.941-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:56:26.941-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:56:26.941-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:56:26.941-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:56:26.941-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:56:26.941-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 is started----------
PipelineManager 2017-04-25T13:56:26.941-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:56:26.941-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:56:26.942-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:56:27.084-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:56:27.089-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:56:27.089-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:56:27.090-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:56:27 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 127 successful
CapiNozzle 2017-04-25T13:56:27.596-05:00 [INFO] Send batch count=185
2017/04/25 13:56:27 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 129 successful
StatisticsManager 2017-04-25T13:56:27.688-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:56:27.700-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:27.700-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:56:27 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 228 successful
CapiNozzle 2017-04-25T13:56:27.720-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:56:27.752-05:00 [INFO] Send batch count=1
2017/04/25 13:56:27 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 26 successful
CapiNozzle 2017-04-25T13:56:27.787-05:00 [INFO] Send batch count=422
2017/04/25 13:56:27 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:56:27 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:56:28 UPR_STREAMREQ for vb 51 successful
CapiNozzle 2017-04-25T13:56:29.021-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:30.785-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
StatisticsManager 2017-04-25T13:56:34.166-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:56:34.071-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:56:34.276-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
2017/04/25 13:56:34 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 155 successful
StatisticsManager 2017-04-25T13:56:34.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:56:34 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:56:34 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 161 successful
CapiNozzle 2017-04-25T13:56:35.166-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:35.192-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:56:35 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 172 successful
CapiNozzle 2017-04-25T13:56:35.348-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
2017/04/25 13:56:35 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 279 successful
CapiNozzle 2017-04-25T13:56:35.366-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
2017/04/25 13:56:35 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 290 successful
StatisticsManager 2017-04-25T13:56:35.680-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:56:35 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 297 successful
CapiNozzle 2017-04-25T13:56:35.768-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:35.780-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:56:35.780-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:56:35.783-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:56:35.807-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
CheckpointManager 2017-04-25T13:56:35.820-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
CheckpointManager 2017-04-25T13:56:35.820-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:56:35 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:56:35 UPR_STREAMREQ for vb 97 successful
DcpNozzle 2017-04-25T13:56:35.855-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
PipelineManager 2017-04-25T13:56:35.855-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:56:35.858-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:56:35.861-05:00 [INFO] err_list=[{"time":"2017-04-25T13:56:35.861238795-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:56:35.926-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
DcpNozzle 2017-04-25T13:56:35.930-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:56:36.011-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:56:36.031-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:56:36.051-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:36.070-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:56:36.099-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
GenericSupervisor 2017-04-25T13:56:36.980-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:56:38.149-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:56:38.556-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:56:38.668-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:56:38.944-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:56:38.958-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:56:38.960-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:56:38.851-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:56:39.047-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
GenericSupervisor 2017-04-25T13:56:40.043-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:56:40.217-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:56:40.200-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:56:41.193-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
GenericSupervisor 2017-04-25T13:56:43.027-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:56:44.170-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:56:45.103-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:56:45.130-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:56:45.039-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:56:35.861238795-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 1000, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 324, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 66332, "docs_rep_queue": 40838, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 26617, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 324, "set_filtered": 0, "set_received_from_dcp": 66332, "size_rep_queue": 53493989, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 50}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 404, "docs_opt_repd": 0, "docs_rep_queue": 47559, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 404, "size_rep_queue": 64288817, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 151, "max": 17, "mean": 16.125827814569536, "min": 16}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 255, "docs_opt_repd": 0, "docs_rep_queue": 42200, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 255, "size_rep_queue": 53654549, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 353, "max": 66, "mean": 64.86685552407933, "min": 34}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 34778, "max": 2962, "mean": 14.737, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 48244, "expiry_received_from_dcp": 0, "set_received_from_dcp": 48244}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 1000, "dcp_dispatch_time": {"count": 31554, "max": 876, "mean": 14.608, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 42730, "expiry_received_from_dcp": 0, "set_received_from_dcp": 42730}}

StatisticsManager 2017-04-25T13:56:45.143-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 49290 items, sent 685 items
StatisticsManager 2017-04-25T13:56:45.143-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 44168 items, sent 923 items
StatisticsManager 2017-04-25T13:56:45.151-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:44321]
StatisticsManager 2017-04-25T13:56:45.153-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 44321 items, sent 44321 items.
StatisticsManager 2017-04-25T13:56:45.154-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:49290]
StatisticsManager 2017-04-25T13:56:45.154-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 49290 items, sent 49290 items.
StatisticsManager 2017-04-25T13:56:45.255-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:56:45.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:56:45.257-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:56:45.275-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 {"Errors": "[{\"time\":\"2017-04-25T13:56:35.861238795-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 0, "dcp_datach_length": 1000, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 324, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 2, "docs_received_from_dcp": 66332, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 324, "set_filtered": 0, "set_received_from_dcp": 66332, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

PipelineManager 2017-04-25T13:56:46.194-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
GenericSupervisor 2017-04-25T13:56:47.146-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:56:48.447-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:56:47.283-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:49.892-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:56:49.979-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:56:49.983-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:56:49.754-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:56:50.020-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
PipelineRuntimeCtx 2017-04-25T13:56:50.078-05:00 [INFO] Pipeline context is stopping...
StatisticsManager 2017-04-25T13:56:50.081-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:56:50.083-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:56:50.084-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:56:50.092-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:56:50.103-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:56:50.153-05:00 [INFO] monitorPipelineHealth routine is exiting because parent supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata has been stopped
GenericSupervisor 2017-04-25T13:56:50.154-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:50.09238737 -0500 CDT
GenericSupervisor 2017-04-25T13:56:50.337-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:56:50.337-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:56:50.349-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:56:50.349-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:56:50.408-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:56:50.408-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineRuntimeCtx 2017-04-25T13:56:50.417-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 has been stopped
CapiNozzle 2017-04-25T13:56:50.418-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:50.555-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericPipeline 2017-04-25T13:56:50.604-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-420327248 is stopped
PipelineManager 2017-04-25T13:56:50.604-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
AsyncListener 2017-04-25T13:56:50.635-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:50.649-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:50.649-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:50.649-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:50.649-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:50.649-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:50.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:50.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:56:50.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:50.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:50.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:50.651-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:56:50.668-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:50.668-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.668-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:50.690-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.691-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:56:50.652-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:56:50.725-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:56:50.633-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
GenericPipeline 2017-04-25T13:56:50.634-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
GenericPipeline 2017-04-25T13:56:50.750-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:56:50.754-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:56:50.758-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:56:50.634-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
GenericPipeline 2017-04-25T13:56:50.769-05:00 [INFO] About to call function
PipelineManager 2017-04-25T13:56:50.608-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:56:35.861238795-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

CapiNozzle 2017-04-25T13:56:50.769-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:56:50.774-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
DcpNozzle 2017-04-25T13:56:50.774-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
GenericPipeline 2017-04-25T13:56:50.634-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-420327248
GenericPipeline 2017-04-25T13:56:50.774-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:56:50.774-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:56:50.774-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:56:50.774-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:50.774127343 -0500 CDT
GenericPipeline 2017-04-25T13:56:50.778-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:56:50.778-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:56:50.778-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:56:50.778-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
PipelineManager 2017-04-25T13:56:50.785-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:56:50.786-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:56:50.786-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:56:50.786-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:50.778438471 -0500 CDT
CapiNozzle 2017-04-25T13:56:50.812-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:56:50.812-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
GenericPipeline 2017-04-25T13:56:50.812-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:50.815-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
CapiNozzle 2017-04-25T13:56:50.820-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:56:50.928-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:56:50.928-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:56:50.928-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:50.929-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
DcpNozzle 2017-04-25T13:56:51.076-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:56:51.076-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:51.076211556 -0500 CDT
DcpNozzle 2017-04-25T13:56:51.076-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:56:51.076-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:51.076-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:56:51.082-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:56:51.082-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:56:51.082538032 -0500 CDT
DcpNozzle 2017-04-25T13:56:51.082-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:56:51.082-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:56:51.082-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
PipelineManager 2017-04-25T13:56:51.125-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:56:35.861238795-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Start pipeline construction}
]
ReplicationManager 2017-04-25T13:56:51.753-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=2, changes_left=315506
XDCRFactory 2017-04-25T13:56:51.765-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:56:51.774-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:56:51.774-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:56:51.923-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:56:51.925-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:56:51.939-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:56:51.939-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:56:51.939-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:56:51.939-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:56:51.939-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:56:51.942-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:56:51.946-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
GenericSupervisor 2017-04-25T13:56:51.951-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:56:51.951-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
GenericSupervisor 2017-04-25T13:56:51.951-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:51.951-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:51.951-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:51.951-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:56:51.951-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:56:51.955-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
CheckpointManager 2017-04-25T13:56:51.955-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
CapiUtils 2017-04-25T13:56:52.051-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:56:52.052-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:56:52.053-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:56:52.056-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
StatisticsManager 2017-04-25T13:56:52.068-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:56:52.069-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
XDCRFactory 2017-04-25T13:56:52.069-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:56:52.070-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343 is constructed, start it
GenericPipeline 2017-04-25T13:56:52.072-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[60 107 43 79 86 92 115 134 1 63 70 97 110 4 145 33 55 78 96 138 76 109 89 73 9 15 111 137 147 101 148 165 26 84 85 13 16 57 160 31 50 100 39 94 103 105 119 151 11 166 141 54 153 21 82 17 46 80 6 72 116 129 38 130 65 118 136 2 28 22 24 51 112 123 125 7 23 69 104 127 140 170 41 99 146 35 90 159 5 108 149 40 139 48 44 154 169 14 66 102 45 132 168 56 93 37 75 128 163 67 122 150 18 106 117 142 161 71 77 133 143 152 36 59 113 126 10 20 87 95 121 164 3 49 131 42 68 81 144 162 120 156 91 157 0 83 52 74 88 114 124 29 62 64 47 58 155 167 8 25 135 98 12 19 27 34 158 61 32 30 53] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[221 298 333 213 256 263 269 278 313 197 229 316 191 336 177 240 182 185 225 266 286 258 280 174 207 275 300 311 332 340 192 204 218 293 259 320 220 282 230 241 274 179 187 217 249 199 216 288 303 312 222 242 252 285 330 224 235 309 296 297 247 212 267 290 314 172 326 270 277 339 233 319 193 196 251 276 271 180 181 203 328 186 283 301 261 262 226 264 202 223 232 201 255 289 323 194 208 206 236 183 238 324 260 322 265 337 228 244 304 310 317 272 173 291 305 318 321 190 295 338 188 234 178 287 331 231 327 284 306 215 248 302 214 315 308 175 211 250 273 334 329 198 246 200 227 294 335 184 245 292 210 268 279 176 237 257 281 307 325 171 195 205 239 243 254 299 189 209 219 253]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[41 99 146 35 90 159 5 108 149 40 139 48 44 154 169 14 66 102 45 132 168 56 93 37 75 128 163 67 122 150 161 18 106 117 142 152 71 77 133 143 36 59 113 126 121 164 10 20 87 95 3 49 131 162 42 68 81 144 120 156 91 157 0 83 124 52 74 88 114 29 62 64 47 58 155 167 8 25 135 98 158 12 19 27 34 61 32 30 53 60 107 115 134 43 79 86 92 1 63 70 97 110 4 145 138 33 55 78 96 76 109 89 73 147 9 15 111 137 101 148 165 26 84 85 13 16 57 160 31 50 100 119 151 39 94 103 105 11 166 141 54 153 21 82 17 46 80 6 72 116 129 38 130 65 118 136 2 28 123 125 22 24 51 112 7 23 69 104 127 140 170] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[323 289 208 194 236 206 324 183 238 260 322 337 265 310 317 228 244 304 272 305 173 291 190 295 318 321 338 178 188 234 287 331 231 327 284 306 248 302 215 214 315 250 273 308 175 211 329 334 246 198 200 227 245 292 294 335 184 210 268 176 279 237 257 281 239 243 307 325 171 195 205 219 253 254 299 189 209 333 221 298 213 256 263 269 278 313 229 316 197 191 336 177 240 182 185 225 266 286 174 258 280 300 207 275 311 332 340 192 204 218 293 220 259 320 241 274 282 230 187 179 199 216 217 249 303 312 288 222 242 252 285 330 309 224 235 296 297 247 290 314 212 267 172 326 277 270 233 319 339 251 276 193 196 271 328 180 181 203 186 283 301 261 262 226 264 223 232 202 201 255]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[source_nozzle_per_node:2 log_level:Info doc_batch_size_kb:2048 optimistic_replication_threshold:256 active:true checkpoint_interval:1800 worker_batch_size:500 failure_restart_interval:10 target_nozzle_per_node:2 stats_interval:1000 replication_type:capi filter_expression:]
CheckpointManager 2017-04-25T13:56:52.075-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343...
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:56:52.121-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
StatisticsManager 2017-04-25T13:56:52.123-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:56:52.129-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:56:52.130-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:56:52.130-05:00 [INFO] Service TopologyChangeDetectSvc has been started
GenericSupervisor 2017-04-25T13:56:52.130-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:56:52.130-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:56:52.130-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:56:52.131-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:56:52.134-05:00 [INFO] Checkpointing starts in 1802.069 sec
PipelineRuntimeCtx 2017-04-25T13:56:52.134-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:56:52.134-05:00 [INFO] updateStats started
GenericSupervisor 2017-04-25T13:56:52.139-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:56:52.139-05:00 [INFO] checkpointing rountine started
CapiNozzle 2017-04-25T13:56:52.144-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:56:52.172-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:56:52.172-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:56:52.172-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:56:52.192-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:56:52.192-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:56:52.199-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:56:52.220-05:00 [INFO] validateSourceTopology...
CapiUtils 2017-04-25T13:56:52.246-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:56:52.246-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:56:52.247-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:56:52.285-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:56:52.362-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:56:52.362-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:56:52.362-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:56:52.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:56:52.366-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:56:52.367-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:56:52.392-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:56:52.392-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:56:52.392-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:56:52.422-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:56:52.422-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:56:52.491-05:00 [INFO] Got the bucket metadata for ckmgr
DcpNozzle 2017-04-25T13:56:52.598-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:56:52.598-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:56:52.598-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:56:52.598-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:56:52.598-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:56:52.619-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:56:52.619-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343 is started----------
PipelineManager 2017-04-25T13:56:52.619-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:56:52.619-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:56:52.620-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:56:52.801-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:56:52.806-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:56:52.806-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:56:52.807-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:56:52 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:56:52 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:56:52 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:56:52 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:56:53.091-05:00 [INFO] Send batch count=66
2017/04/25 13:56:53 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 4 successful
CapiNozzle 2017-04-25T13:56:53.099-05:00 [INFO] Send batch count=44
2017/04/25 13:56:53 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 207 successful
CapiNozzle 2017-04-25T13:56:53.164-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:56:53.164-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:56:53 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 107 successful
StatisticsManager 2017-04-25T13:56:53.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:56:53.242-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:56:53 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 126 successful
CapiNozzle 2017-04-25T13:56:53.490-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:56:53.508-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:56:53 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 131 successful
CapiNozzle 2017-04-25T13:56:53.609-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
2017/04/25 13:56:53 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 333 successful
CapiNozzle 2017-04-25T13:56:53.627-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:56:53 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:56:53 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 261 successful
StatisticsManager 2017-04-25T13:56:54.177-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:56:54.185-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:56:54 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 65 successful
CapiNozzle 2017-04-25T13:56:54.200-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
GenericSupervisor 2017-04-25T13:56:54.213-05:00 [ERROR] Heartbeat timeout in supervisor PipelineMasterSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:56:54.213-05:00 [INFO] Child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of supervisor PipelineMasterSupervisor missed 1 consecutive heart beats
2017/04/25 13:56:54 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 199 successful
CheckpointManager 2017-04-25T13:56:54.640-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
CheckpointManager 2017-04-25T13:56:54.640-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:56:54 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:56:54 UPR_STREAMREQ for vb 296 successful
DcpNozzle 2017-04-25T13:56:54.773-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:56:54.810-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
CapiNozzle 2017-04-25T13:56:54.841-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:56:54.853-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:56:54.907-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:56:54.918-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:56:55.713-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:56:55.776-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:56:55.776-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:56:55.742-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:56:55.850-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:56:55.850-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:56:55.779-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
ReplicationManager 2017-04-25T13:56:55.935-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]
GenericSupervisor 2017-04-25T13:56:55.850-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:56:56.620-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:56:58.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
StatisticsManager 2017-04-25T13:56:59.900-05:00 [INFO] Pipeline is no longer running, exit.
PipelineManager 2017-04-25T13:57:01.641-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
GenericSupervisor 2017-04-25T13:57:05.165-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:57:05.178-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
ToplogyChangeDetector 2017-04-25T13:57:05.043-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:57:05.068-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:57:05.277-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:57:05.278-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:57:05.261-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:57:05.355-05:00 [INFO] Finish executing 0x681fb0
GenericSupervisor 2017-04-25T13:57:06.263-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
PipelineManager 2017-04-25T13:57:05.047-05:00 [INFO] err_list=[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]
GenericSupervisor 2017-04-25T13:57:06.449-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:57:09.883-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:57:10.144-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
StatisticsManager 2017-04-25T13:57:10.146-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343 {"Errors": "[]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 67, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 47, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 55657, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 47, "set_filtered": 0, "set_received_from_dcp": 55657, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Pipeline is running", "Status": "Pending"}

PipelineManager 2017-04-25T13:57:10.163-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:57:10.172-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:57:10.342-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:57:10.495-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:57:10.927-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
PipelineManager 2017-04-25T13:57:11.678-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries]}
]
CapiNozzle 2017-04-25T13:57:12.066-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:57:12.199-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:57:12.335-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CheckpointManager 2017-04-25T13:58:14.374-05:00 [INFO] Pipeline is no longer running, exit.
CheckpointManager 2017-04-25T13:58:14.380-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
PipelineManager 2017-04-25T13:58:14.356-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries 
PipelineManager 2017-04-25T13:58:14.461-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:58:14.463-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:58:14.465-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
PipelineRuntimeCtx 2017-04-25T13:58:14.468-05:00 [INFO] Pipeline context is stopping...
CapiNozzle 2017-04-25T13:58:14.231-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:58:14.539-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
GenericSupervisor 2017-04-25T13:58:14.471-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:58:14.552-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:58:14.554-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:58:14.547986779 -0500 CDT
GenericSupervisor 2017-04-25T13:58:14.577-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:58:14.578-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:58:14.579-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:58:14.579-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:58:14.580-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:58:14.580-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:58:14.581-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:58:14.581-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-939660343 has been stopped
GenericPipeline 2017-04-25T13:58:14.653-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-939660343 is stopped
PipelineManager 2017-04-25T13:58:14.653-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:58:14.655-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:58:14.660-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:58:14.670-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
GenericPipeline 2017-04-25T13:58:14.675-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
GenericPipeline 2017-04-25T13:58:14.675-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
GenericPipeline 2017-04-25T13:58:14.675-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-939660343
AsyncListener 2017-04-25T13:58:14.677-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:58:14.686-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:58:14.686-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:58:14.686-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:58:14.687-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
GenericPipeline 2017-04-25T13:58:14.687-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:58:14.688-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:58:14.691-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:58:14.702-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:58:14.704-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:58:14.706-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:58:14.706-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:58:14.706-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:58:14.706-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
GenericPipeline 2017-04-25T13:58:14.706-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:58:14.706-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:58:14.706-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
AsyncListener 2017-04-25T13:58:14.706-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.706-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.706-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.706-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:58:14.707-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:58:14.707-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:58:14.707-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:58:14.70634672 -0500 CDT
CapiNozzle 2017-04-25T13:58:14.709-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:58:14.706918757 -0500 CDT
CapiNozzle 2017-04-25T13:58:14.709-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:58:14.709-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:58:14.709-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.714-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:58:14.714-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:58:14.716-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:58:14.716-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.716-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:58:14.716-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:58:14.717-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:58:14.740-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
CapiNozzle 2017-04-25T13:58:14.740-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
DcpNozzle 2017-04-25T13:58:14.880-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:58:14.880-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:58:14.880714781 -0500 CDT
DcpNozzle 2017-04-25T13:58:14.880-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:58:14.880-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:58:14.881-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
DcpNozzle 2017-04-25T13:58:14.931-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:58:14.932-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:58:14.932467623 -0500 CDT
DcpNozzle 2017-04-25T13:58:14.932-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:58:14.932-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:58:14.932-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
CapiNozzle 2017-04-25T13:58:15.132-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:58:15.136-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:58:15.154-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:58:15.154-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
PipelineManager 2017-04-25T13:58:15.268-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={Start pipeline construction}
]
CapiNozzle 2017-04-25T13:58:15.469-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:58:15.469-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:58:15.469-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:58:15.469-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
XDCRFactory 2017-04-25T13:58:15.489-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:58:15.501-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:58:15.503-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
PipelineManager 2017-04-25T13:58:15.642-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={2 source nozzles have been constructed}
]
XDCRFactory 2017-04-25T13:58:15.918-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:58:15.925-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:58:15.929-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:58:15.929-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:58:15.929-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:58:15.929-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:58:15.929-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ReplicationManager 2017-04-25T13:58:15.931-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=4, changes_left=315504
XDCRFactory 2017-04-25T13:58:15.934-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:58:15.940-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
GenericSupervisor 2017-04-25T13:58:15.959-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:58:15.959-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
GenericSupervisor 2017-04-25T13:58:15.960-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:58:15.960-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:58:15.960-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:58:15.960-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:58:15.960-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:58:15.963-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
CheckpointManager 2017-04-25T13:58:15.963-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
CapiUtils 2017-04-25T13:58:16.085-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:58:16.086-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:58:16.087-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:58:16.092-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
StatisticsManager 2017-04-25T13:58:16.101-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:58:16.101-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
XDCRFactory 2017-04-25T13:58:16.102-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:58:16.102-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 is constructed, start it
GenericPipeline 2017-04-25T13:58:16.114-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[193 194 249 278 255 293 210 232 327 175 267 274 319 216 221 177 189 280 310 326 338 256 289 187 252 245 333 281 260 192 195 211 219 272 212 229 172 237 292 300 174 198 296 182 214 283 298 330 286 234 257 190 311 266 277 321 231 242 282 209 240 323 337 200 213 253 334 220 313 317 191 295 208 285 270 246 262 265 269 251 307 339 196 206 316 247 340 318 181 186 235 315 324 233 241 312 332 224 258 178 202 301 309 329 199 203 236 302 204 238 259 314 183 222 275 297 303 184 215 226 279 185 248 305 328 188 243 250 263 322 171 179 276 180 225 173 254 320 325 273 223 227 228 331 176 264 306 201 290 218 230 244 268 205 239 287 291 299 336 217 271 288 304 308 207 261 284 197 294 335] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[31 44 79 94 96 130 24 82 127 105 106 87 89 119 147 4 39 40 68 69 104 140 92 99 132 139 63 100 110 112 98 102 151 164 75 93 108 125 142 95 121 143 152 156 54 166 5 28 74 154 65 122 153 45 83 109 114 10 36 167 14 15 47 30 20 51 56 57 124 136 25 131 0 9 53 84 71 22 26 34 64 66 67 81 117 170 80 129 135 18 41 116 134 38 141 55 168 163 6 8 113 149 145 137 146 165 13 59 77 32 103 111 120 43 158 7 61 23 133 49 76 62 155 42 50 52 160 12 118 144 169 3 123 150 46 73 90 91 159 35 37 86 2 72 88 21 33 107 126 17 48 58 78 157 162 97 85 101 115 1 19 161 29 16 27 60 70 128 138 11 148]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[192 195 211 219 260 212 229 272 172 237 300 292 296 174 198 283 298 330 182 214 286 234 257 311 190 266 277 321 231 242 282 209 337 240 323 200 213 253 334 220 313 317 191 295 208 285 246 262 265 269 270 307 339 251 206 196 316 340 247 181 186 235 318 233 241 312 315 324 332 178 202 224 258 301 309 199 203 236 302 329 204 238 259 183 222 275 297 314 184 215 226 279 303 185 248 305 328 188 243 250 263 322 171 179 276 173 180 225 325 254 320 223 227 228 273 176 264 306 331 201 290 218 230 244 268 239 287 205 291 299 336 217 271 288 304 308 207 261 284 197 294 335 193 194 249 278 255 293 210 232 327 267 274 319 175 177 189 216 221 280 310 326 338 187 252 256 289 245 281 333] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[54 166 74 154 5 28 65 122 153 109 45 83 114 167 10 36 47 14 15 30 56 57 124 136 20 51 25 131 53 84 0 9 34 64 66 67 71 22 26 170 81 117 135 80 129 116 134 18 41 38 141 55 168 163 6 149 8 113 145 165 137 146 77 13 59 111 120 32 103 43 158 7 61 23 133 49 76 62 155 52 160 42 50 12 118 144 169 150 3 123 90 91 159 46 73 86 35 37 88 2 72 107 126 21 33 58 78 157 162 17 48 97 115 85 101 161 1 19 29 60 70 16 27 128 138 11 148 79 31 44 130 94 96 127 24 82 105 106 119 87 89 40 68 69 104 147 4 39 140 132 139 92 99 63 112 100 110 151 164 98 102 108 125 142 75 93 143 152 156 95 121]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[optimistic_replication_threshold:256 target_nozzle_per_node:2 log_level:Info failure_restart_interval:10 filter_expression: active:true checkpoint_interval:1800 worker_batch_size:500 doc_batch_size_kb:2048 source_nozzle_per_node:2 stats_interval:1000 replication_type:capi]
CheckpointManager 2017-04-25T13:58:16.116-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453...
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:58:16.150-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:58:16.151-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:58:16.151-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:58:16.151-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:58:16.151-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:58:16.151-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:58:16.152-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:58:16.153-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:58:16.153-05:00 [INFO] Service PipelineSupervisor has been started
CheckpointManager 2017-04-25T13:58:16.156-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:58:16.162-05:00 [INFO] Checkpointing starts in 1800.08 sec
PipelineRuntimeCtx 2017-04-25T13:58:16.162-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:58:16.162-05:00 [INFO] StatisticsManager Starting...
GenericSupervisor 2017-04-25T13:58:16.162-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:58:16.163-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:58:16.176-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:58:16.177-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:58:16.177-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:58:16.178-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:58:16.203-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiUtils 2017-04-25T13:58:16.265-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:58:16.265-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:58:16.269-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:58:16.291-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:58:16.458-05:00 [INFO] Got the bucket metadata for ckmgr
ToplogyChangeDetector 2017-04-25T13:58:17.648-05:00 [INFO] validateSourceTopology...
CheckpointManager 2017-04-25T13:58:19.460-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:58:19.473-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:58:19.478-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:58:19.433-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
StatisticsManager 2017-04-25T13:58:19.530-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:20.104-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:20.470-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:22.891-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:23.520-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:23.803-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:24.575-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
PipelineManager 2017-04-25T13:58:25.358-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={The runtime context is started}
]
StatisticsManager 2017-04-25T13:58:25.739-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:58:26.166-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:58:26.201-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:57:05.045685176-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:58:26.201-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:58:26.201-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:58:26.204-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:58:26.208-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:58:26.208-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:58:26.208-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:26.209-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:26.762-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:58:27.263-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:58:27.636-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:58:27.651-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:58:29.684-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ToplogyChangeDetector 2017-04-25T13:58:30.209-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:58:30.944-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:32.894-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:33.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:34.828-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:35.466-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:35.782-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
GenericSupervisor 2017-04-25T13:58:36.167-05:00 [INFO] About to call function
StatisticsManager 2017-04-25T13:58:36.564-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:36.570-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:57:05.045685176-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315508, "data_replicated": 0, "dcp_datach_length": 0, "dcp_dispatch_time": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 0, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 0, "docs_received_from_dcp": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 0, "set_filtered": 0, "set_received_from_dcp": 0, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "The runtime context is started", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Pending", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 0, "docs_opt_repd": 0, "docs_rep_queue": 0, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 0, "size_rep_queue": 0, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 0, "dcp_dispatch_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 0, "expiry_received_from_dcp": 0, "set_received_from_dcp": 0}}

StatisticsManager 2017-04-25T13:58:36.571-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:58:36.571-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 0 items, sent 0 items
StatisticsManager 2017-04-25T13:58:36.576-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:0]
StatisticsManager 2017-04-25T13:58:36.580-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:58:36.580-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:0]
StatisticsManager 2017-04-25T13:58:36.580-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 0 items, sent 0 items.
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:36.581-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
GenericSupervisor 2017-04-25T13:58:37.230-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:58:37.231-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:58:37.233-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
StatisticsManager 2017-04-25T13:58:38.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:40.884-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:42.047-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
ReplicationManager 2017-04-25T13:58:42.172-05:00 [INFO] Updating status for paused replication f4268e62702130298bf87f17cc481219/metadata/metadata. kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]], total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:58:42.205-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:58:42.206-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:58:42.210-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:58:42.232-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:58:42.234-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
StatisticsManager 2017-04-25T13:58:42.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
StatisticsManager 2017-04-25T13:58:42.383-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:58:42.469-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:58:42.596-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:58:42.597-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:58:42.632-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:58:42.634-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:58:42.663-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
PipelineManager 2017-04-25T13:58:42.760-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:57:05.045685176-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries"}]}, progress={The runtime context is started}
]
CapiNozzle 2017-04-25T13:58:42.882-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:58:42.882-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:58:42.882-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:58:42.912-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:58:42.913-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:58:43.480-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:58:43.553-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:58:43.556-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
GenericPipeline 2017-04-25T13:58:43.653-05:00 [INFO] All parts has been started
DcpNozzle 2017-04-25T13:58:43.627-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:58:43.627-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
2017/04/25 13:58:43 UPR_STREAMREQ for vb 200 successful
GenericPipeline 2017-04-25T13:58:43.869-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 is started----------
PipelineManager 2017-04-25T13:58:43.878-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:58:43.879-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
2017/04/25 13:58:43 UPR_STREAMREQ for vb 100 successful
PipelineManager 2017-04-25T13:58:43.911-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:58:44 UPR_STREAMREQ for vb 300 successful
CapiNozzle 2017-04-25T13:58:44.065-05:00 [INFO] Send batch count=1
StatisticsManager 2017-04-25T13:58:44.108-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:58:44 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:58:44 UPR_STREAMREQ for vb 201 successful
CapiNozzle 2017-04-25T13:58:44.217-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:58:44.569-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:58:44.605-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:58:44.667-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:58:44 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:58:44 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:58:44 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:58:44 UPR_STREAMREQ for vb 202 successful
StatisticsManager 2017-04-25T13:58:44.759-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:58:44 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:58:44 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:58:45 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 2 successful
GenericSupervisor 2017-04-25T13:58:46.335-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:58:45.524-05:00 [INFO] Send batch count=500
2017/04/25 13:58:46 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 104 successful
StatisticsManager 2017-04-25T13:58:46.837-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
CapiNozzle 2017-04-25T13:58:46.751-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:58:46.846-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:58:46 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:58:46 UPR_STREAMREQ for vb 305 successful
CapiNozzle 2017-04-25T13:58:46.907-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
2017/04/25 13:58:47 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 106 successful
StatisticsManager 2017-04-25T13:58:47.118-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=1, changes_left=315507
StatisticsManager 2017-04-25T13:58:47.121-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:57:05.045685176-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:batch update docs failed for vb 300 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315507, "data_replicated": 252, "dcp_datach_length": 823, "dcp_dispatch_time": 14, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 299, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 1, "docs_received_from_dcp": 16026, "docs_rep_queue": 6880, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 4062, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 299, "set_filtered": 0, "set_received_from_dcp": 16026, "size_rep_queue": 8959486, "time_committing": 0, "wtavg_docs_latency": 1287, "wtavg_meta_latency": 411}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 299, "docs_opt_repd": 0, "docs_rep_queue": 2344, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 299, "size_rep_queue": 2859547, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 409, "max": 413, "mean": 411.80195599022005, "min": 411}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 252, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 314, "docs_opt_repd": 1, "docs_rep_queue": 7839, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 1, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 1, "set_failed_cr_source": 314, "size_rep_queue": 10464008, "wtavg_docs_latency": {"count": 1, "max": 1287, "mean": 1287, "min": 1287}, "wtavg_meta_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 554, "dcp_dispatch_time": {"count": 7686, "max": 1051, "mean": 14.122, "min": 9}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 9168, "expiry_received_from_dcp": 0, "set_received_from_dcp": 9171}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 269, "dcp_dispatch_time": {"count": 8340, "max": 1138, "mean": 14.866, "min": 7}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 8726, "expiry_received_from_dcp": 0, "set_received_from_dcp": 8726}}

StatisticsManager 2017-04-25T13:58:47.121-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 8726 items, sent 501 items
StatisticsManager 2017-04-25T13:58:47.122-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 9410 items, sent 500 items
2017/04/25 13:58:47 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 306 successful
StatisticsManager 2017-04-25T13:58:47.136-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:9410]
StatisticsManager 2017-04-25T13:58:47.138-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 9411 items, sent 9410 items. streams inactive: [51 118 23 131 143 89 136 39 25 47 57 76 117 128 34 81 82 130 142 154 15 161 69 84 86 97 133 138 158 60 165 54 79 121 122 144 146 19 27 83 94 140 26 80 127 129 148 164 48 64 44 20 49 52 63 113 147 9 71 98 111 108 123 145 77 42 55 70 88 157 21 24 87 110 137 152 112 149 153 36 85 99 116 150 12 8 115 132 141 16 28 35 45 67 156 160 14 166 40 66 91 93 151 170 30 41 43 75 120 163 18 168 11 38 68 72 125 159 167 78 95 73 162 124 37 46 53 169 10 114 29 92 155 56 7 31 32 58 61 50 62 109 126 13 33 65 96 134 135 17 59 74 90 119 139 22]
StatisticsManager 2017-04-25T13:58:47.138-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:8726]
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 8726 items, sent 8726 items. streams inactive: [196 245 248 290 318 331 177 192 223 266 284 335 267 292 299 322 340 209 246 310 171 173 187 190 250 283 293 336 172 256 278 285 296 332 175 238 249 279 197 252 257 315 328 214 220 242 316 253 309 215 237 258 289 307 311 234 287 323 211 232 239 241 268 271 275 251 261 334 181 194 298 321 329 176 227 273 317 319 189 207 217 259 182 264 306 183 185 216 270 276 282 244 263 333 180 265 281 337 218 327 179 228 231 255 286 330 272 178 191 213 221 225 247 308 325 339 174 210 226 229 338 212 219 235 254 297 222 277 193 233 274 280 294 313 320 198 230 236 240 260 262 291 195 199 208 224 269 312 324 184 186 188 243 288 295 314 326]
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:58:47.139-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
2017/04/25 13:58:47 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 9 successful
GenericSupervisor 2017-04-25T13:58:47.383-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:58:47.385-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:58:47.385-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:58:47 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 210 successful
StatisticsManager 2017-04-25T13:58:47.397-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:58:47 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:58:47 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 122 successful
StatisticsManager 2017-04-25T13:58:48.519-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:58:48 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:58:48 UPR_STREAMREQ for vb 325 successful
CapiNozzle 2017-04-25T13:58:48.789-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:58:51 UPR_STREAMREQ for vb 124 successful
CapiNozzle 2017-04-25T13:58:51.306-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:58:51.230-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
2017/04/25 13:58:51 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 126 successful
CapiNozzle 2017-04-25T13:58:51.730-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:58:51 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:58:51 UPR_STREAMREQ for vb 28 successful
StatisticsManager 2017-04-25T13:58:51.946-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:58:52 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 230 successful
StatisticsManager 2017-04-25T13:58:52.296-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:58:52 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 30 successful
DcpNozzle 2017-04-25T13:58:52.407-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 incrementing counter for inactive streams 1
2017/04/25 13:58:52 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 32 successful
StatisticsManager 2017-04-25T13:58:52.527-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:58:52 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:58:52 UPR_STREAMREQ for vb 39 successful
CapiNozzle 2017-04-25T13:58:53.044-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:58:53 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 246 successful
CapiNozzle 2017-04-25T13:58:53.108-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
2017/04/25 13:58:53 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 253 successful
CapiNozzle 2017-04-25T13:58:53.290-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:58:53 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 255 successful
StatisticsManager 2017-04-25T13:58:53.341-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
CapiNozzle 2017-04-25T13:58:53.380-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
2017/04/25 13:58:53 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:58:53 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 65 successful
StatisticsManager 2017-04-25T13:58:54.474-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=2, changes_left=315506
2017/04/25 13:58:54 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 68 successful
CapiNozzle 2017-04-25T13:58:54.540-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
2017/04/25 13:58:54 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 274 successful
CapiNozzle 2017-04-25T13:58:54.570-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:58:54.570-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
2017/04/25 13:58:54 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 170 successful
GenericSupervisor 2017-04-25T13:58:54.571-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:58:54.620-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
2017/04/25 13:58:54 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 276 successful
PipelineManager 2017-04-25T13:58:54.685-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:58:54.687-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:58:54.695-05:00 [INFO] err_list=[{"time":"2017-04-25T13:58:54.695231964-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:58:54.746-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
2017/04/25 13:58:54 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 76 successful
CapiNozzle 2017-04-25T13:58:54.803-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:58:54.821-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:58:54.846-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
2017/04/25 13:58:54 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:58:54 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 85 successful
CapiNozzle 2017-04-25T13:58:55.139-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
2017/04/25 13:58:55 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 94 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 190 successful
CapiNozzle 2017-04-25T13:58:55.175-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
StatisticsManager 2017-04-25T13:58:55.209-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:58:55.212-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 {"Errors": "[{\"time\":\"2017-04-25T13:58:54.695231964-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315506, "data_replicated": 252, "dcp_datach_length": 288, "dcp_dispatch_time": 159, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 613, "docs_filtered": 0, "docs_opt_repd": 1, "docs_processed": 2, "docs_received_from_dcp": 66653, "docs_rep_queue": 0, "docs_written": 1, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 1, "set_failed_cr_source": 613, "set_filtered": 0, "set_received_from_dcp": 66653, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

2017/04/25 13:58:55 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 194 successful
CheckpointManager 2017-04-25T13:58:55.253-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
CheckpointManager 2017-04-25T13:58:55.253-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:58:55 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 97 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:58:55 UPR_STREAMREQ for vb 99 successful
PipelineManager 2017-04-25T13:58:55.285-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:58:54.695231964-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]}
]
DcpNozzle 2017-04-25T13:58:55.365-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
2017/04/25 13:58:55 UPR_STREAMREQ for vb 199 successful
DcpNozzle 2017-04-25T13:58:55.441-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
GenericSupervisor 2017-04-25T13:58:56.187-05:00 [INFO] About to call function
GenericSupervisor 2017-04-25T13:58:56.187-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:58:56.187-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:58:56.348-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:58:56.373-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
CapiNozzle 2017-04-25T13:58:56.736-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:58:56.736-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:58:56.736-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:58:56.736-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:58:56.736-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:58:56.871-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:58:56.871-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:58:56.884-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:58:57.422-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:58:57.444-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
CapiNozzle 2017-04-25T13:58:57.882-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:58:57.897-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:58:58.795-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:58:58.795-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 0 after 3 retries
CapiNozzle 2017-04-25T13:58:58.795-05:00 [ERROR] Raise error condition batch update docs failed for vb 0 after 3 retries
GenericSupervisor 2017-04-25T13:58:58.795-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:58:58.795-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:58:58.885-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:58:58.886-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:58:58.900-05:00 [INFO] Retrying update docs for vb 101 for the 1th time
CapiNozzle 2017-04-25T13:58:59.436-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:58:59.451-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:59:00.964-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:59:00.986-05:00 [INFO] Retrying update docs for vb 101 for the 2th time
CapiNozzle 2017-04-25T13:59:01.949-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:59:01.950-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:59:01.950-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:59:01.950-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:59:01.950-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:59:02.057-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:02.057-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:02.068-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:59:02.147-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:59:02.158-05:00 [INFO] Retrying update docs for vb 101 for the 3th time
CapiNozzle 2017-04-25T13:59:02.873-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:59:02.884-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:59:03.131-05:00 [ERROR] batchUpdateDocs for vb 101 failed with err receieved unexpected status code, 500, from update docs request for vb 101
.
CapiNozzle 2017-04-25T13:59:03.131-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 101 after 3 retries
CapiNozzle 2017-04-25T13:59:03.131-05:00 [ERROR] Raise error condition batch update docs failed for vb 101 after 3 retries
GenericSupervisor 2017-04-25T13:59:03.131-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:59:03.131-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:59:03.285-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:03.285-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:03.296-05:00 [INFO] Retrying update docs for vb 1 for the 1th time
CapiNozzle 2017-04-25T13:59:03.809-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:59:03.821-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
CapiNozzle 2017-04-25T13:59:04.199-05:00 [ERROR] batchUpdateDocs for vb 1 failed with err receieved unexpected status code, 500, from update docs request for vb 1
.
CapiNozzle 2017-04-25T13:59:04.210-05:00 [INFO] Retrying update docs for vb 1 for the 2th time
CapiNozzle 2017-04-25T13:59:04.481-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:59:04.481-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 201 after 3 retries
CapiNozzle 2017-04-25T13:59:04.481-05:00 [ERROR] Raise error condition batch update docs failed for vb 201 after 3 retries
GenericSupervisor 2017-04-25T13:59:04.481-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:59:04.481-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:59:04.565-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:04.565-05:00 [ERROR] batchUpdateDocs for vb 301 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:04.576-05:00 [INFO] Retrying update docs for vb 301 for the 1th time
PipelineManager 2017-04-25T13:59:04.746-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:59:04.747-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:59:04.747-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:59:04.748-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
PipelineRuntimeCtx 2017-04-25T13:59:04.749-05:00 [INFO] Pipeline context is stopping...
GenericSupervisor 2017-04-25T13:59:04.750-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:59:04.750-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
GenericSupervisor 2017-04-25T13:59:04.751-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:04.750292306 -0500 CDT
GenericSupervisor 2017-04-25T13:59:04.754-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:59:04.754-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
CheckpointManager 2017-04-25T13:59:04.756-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:59:04.756-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:59:04.756-05:00 [INFO] Exits checkpointing routine.
CheckpointManager 2017-04-25T13:59:04.756-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
StatisticsManager 2017-04-25T13:59:04.756-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:59:04.756-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:59:04.756-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
PipelineRuntimeCtx 2017-04-25T13:59:04.756-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 has been stopped
GenericPipeline 2017-04-25T13:59:04.757-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-929501453 is stopped
PipelineManager 2017-04-25T13:59:04.757-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:59:04.757-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:58:54.695231964-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:59:04.757-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:59:04.758-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
GenericPipeline 2017-04-25T13:59:04.758-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
GenericPipeline 2017-04-25T13:59:04.759-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
GenericPipeline 2017-04-25T13:59:04.759-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-929501453
GenericPipeline 2017-04-25T13:59:04.759-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:04.759-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
CapiNozzle 2017-04-25T13:59:04.760-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:59:04.760-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:59:04.761-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
AsyncListener 2017-04-25T13:59:04.761-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:04.763-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:04.764-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:59:04.762-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
GenericPipeline 2017-04-25T13:59:04.765-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:04.765-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
CapiNozzle 2017-04-25T13:59:04.766-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
GenericPipeline 2017-04-25T13:59:04.766-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:59:04.766-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
DcpNozzle 2017-04-25T13:59:04.766-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CapiNozzle 2017-04-25T13:59:04.766-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:04.760851772 -0500 CDT
DcpNozzle 2017-04-25T13:59:04.766-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
AsyncListener 2017-04-25T13:59:04.766-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:04.766-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:04.76615784 -0500 CDT
DcpNozzle 2017-04-25T13:59:04.766-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:59:04.767-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] Finish executing 0x793f70
CapiNozzle 2017-04-25T13:59:04.767-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.767-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
CapiNozzle 2017-04-25T13:59:04.767-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:59:04.767-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
CapiNozzle 2017-04-25T13:59:04.767-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:59:04.767-05:00 [INFO] releasing capi client
GenericPipeline 2017-04-25T13:59:04.768-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:04.768-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:59:04.768-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:04.768-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:59:04.772-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:59:04.772-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:59:04.772-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:04.772132518 -0500 CDT
DcpNozzle 2017-04-25T13:59:04.772-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:04.77215765 -0500 CDT
DcpNozzle 2017-04-25T13:59:04.772-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
DcpNozzle 2017-04-25T13:59:04.772-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:04.772-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:04.772-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
2017/04/25 13:59:04 Stream Closed for vb 171 StreamEnd simulated
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:04.772-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
2017/04/25 13:59:04 Stream Closed for vb 0 StreamEnd simulated
XDCRFactory 2017-04-25T13:59:04.909-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:59:04.911-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:59:04.911-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:59:05.017-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:59:05.019-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:59:05.022-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:59:05.022-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:59:05.022-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:59:05.022-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:59:05.022-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:59:05.023-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:59:05.026-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
GenericSupervisor 2017-04-25T13:59:05.030-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:59:05.030-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
GenericSupervisor 2017-04-25T13:59:05.030-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:05.030-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:05.030-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:05.030-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:05.030-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:59:05.034-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
CheckpointManager 2017-04-25T13:59:05.034-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
CapiUtils 2017-04-25T13:59:05.108-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:59:05.108-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:59:05.108-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:59:05.111-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
StatisticsManager 2017-04-25T13:59:05.114-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:59:05.115-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
XDCRFactory 2017-04-25T13:59:05.115-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:59:05.116-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865 is constructed, start it
GenericPipeline 2017-04-25T13:59:05.118-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[126 3 8 28 81 156 32 62 127 153 15 59 114 116 84 103 167 12 25 48 52 56 132 144 30 6 100 1 119 161 27 105 91 96 122 136 9 21 115 149 14 61 74 98 60 68 104 13 110 5 16 112 147 160 23 36 134 75 123 128 129 109 145 79 95 113 47 162 111 38 46 138 67 83 125 97 19 55 69 78 50 90 141 120 133 139 159 40 42 73 106 43 49 24 58 101 102 10 17 94 0 130 150 31 33 99 148 22 92 154 89 121 155 39 63 143 146 65 157 7 26 54 166 70 72 118 165 163 37 51 71 124 80 108 158 169 87 107 93 168 11 45 117 2 64 77 82 35 41 4 142 76 131 170 44 57 18 137 152 20 34 66 140 135 164 151 29 53 85 88 86] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[177 197 239 313 255 281 314 220 225 241 185 208 323 299 336 198 199 230 260 171 186 244 321 271 273 330 338 301 227 261 310 174 269 189 263 278 286 249 188 218 290 231 276 304 340 207 176 228 272 317 264 289 339 183 254 180 242 335 270 287 234 258 280 331 192 223 187 245 257 191 203 224 236 265 193 324 222 235 266 295 172 315 291 297 181 194 250 178 205 196 296 204 210 292 217 303 307 274 229 262 288 216 246 212 326 328 211 219 221 202 215 306 190 311 284 300 302 247 268 318 334 173 285 305 333 184 201 267 327 325 237 252 213 226 175 182 243 293 240 320 316 277 279 298 308 214 259 319 232 179 253 282 332 200 209 322 309 275 294 312 337 251 329 206 233 248 256 283 195 238]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[67 83 125 19 55 69 78 97 50 90 141 133 139 159 40 42 73 106 120 43 49 24 58 101 102 10 17 94 0 130 31 33 99 148 150 22 92 154 89 121 155 39 63 143 146 65 157 7 26 54 70 72 118 165 166 37 51 71 124 163 80 108 158 169 87 107 93 168 11 45 117 2 64 77 82 35 41 4 142 76 131 170 44 57 18 137 152 20 34 66 140 135 164 29 53 85 88 151 86 3 8 28 81 126 156 32 62 127 15 59 114 116 153 103 167 12 25 48 52 84 56 132 144 30 6 100 1 119 161 27 105 91 96 122 136 9 21 115 14 61 74 98 149 60 68 104 13 110 5 16 112 147 160 23 36 75 123 128 129 134 109 145 79 95 113 47 162 111 38 46 138] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[196 205 204 210 292 296 217 303 307 274 262 288 229 246 212 216 326 328 211 219 202 221 306 215 190 311 300 302 247 284 268 318 334 173 285 305 333 201 267 327 184 237 252 325 226 175 182 213 293 240 243 320 316 279 298 308 214 259 277 319 179 232 253 282 200 209 322 332 309 294 312 337 275 329 206 233 251 248 256 283 195 238 177 197 239 313 281 314 220 225 241 255 185 208 323 299 336 198 199 230 260 171 186 244 271 321 273 330 338 227 261 301 174 269 310 189 263 278 286 188 249 218 290 276 304 340 207 231 176 228 272 317 264 289 339 254 180 183 242 335 270 287 258 280 331 192 223 234 187 245 257 191 203 224 236 265 193 324 235 266 295 172 222 315 297 181 194 250 291 178]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: checkpoint_interval:1800 worker_batch_size:500 stats_interval:1000 log_level:Info active:true doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:59:05.119-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865...
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:05.142-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
GenericSupervisor 2017-04-25T13:59:05.142-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:59:05.143-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:59:05.143-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:59:05.143-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:59:05.144-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:59:05.144-05:00 [INFO] Checkpointing starts in 1801.547 sec
PipelineRuntimeCtx 2017-04-25T13:59:05.144-05:00 [INFO] Service CheckpointManager has been started
StatisticsManager 2017-04-25T13:59:05.144-05:00 [INFO] StatisticsManager Starting...
CheckpointManager 2017-04-25T13:59:05.145-05:00 [INFO] checkpointing rountine started
PipelineRuntimeCtx 2017-04-25T13:59:05.147-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:59:05.147-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:59:05.147-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:59:05.147-05:00 [INFO] updateStats started
CapiNozzle 2017-04-25T13:59:05.148-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:59:05.176-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:59:05.176-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:59:05.176-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
StatisticsManager 2017-04-25T13:59:05.184-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:59:05.187-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:59:05.187-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:59:05.202-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:59:05.202-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:59:05.202-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
DcpNozzle 2017-04-25T13:59:05.205-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:59:05.205-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
CapiNozzle 2017-04-25T13:59:05.205-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CapiNozzle 2017-04-25T13:59:05.221-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:59:05.221-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:59:05.221-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
ToplogyChangeDetector 2017-04-25T13:59:05.228-05:00 [INFO] validateSourceTopology...
DcpNozzle 2017-04-25T13:59:05.257-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:59:05.257-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CapiUtils 2017-04-25T13:59:05.269-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:59:05.269-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:59:05.269-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CheckpointManager 2017-04-25T13:59:05.321-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
DcpNozzle 2017-04-25T13:59:05.350-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:59:05.350-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:59:05.350-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:59:05.350-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:59:05.350-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
GenericPipeline 2017-04-25T13:59:05.350-05:00 [INFO] All parts has been started
GenericPipeline 2017-04-25T13:59:05.350-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865 is started----------
PipelineManager 2017-04-25T13:59:05.350-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:59:05.350-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:59:05.350-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
CheckpointManager 2017-04-25T13:59:05.475-05:00 [INFO] Got the bucket metadata for ckmgr
CheckpointManager 2017-04-25T13:59:05.714-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
CheckpointManager 2017-04-25T13:59:05.714-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:59:05.714-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:59:05.714-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
2017/04/25 13:59:05 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 201 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 301 successful
CapiNozzle 2017-04-25T13:59:05.822-05:00 [INFO] Send batch count=1
CapiNozzle 2017-04-25T13:59:05.831-05:00 [ERROR] Received malformed response from tcp connection vb=100. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f301/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}

CapiNozzle 2017-04-25T13:59:05.831-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err Received malformed response from tcp connection vb=100. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f301/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] m}
.
2017/04/25 13:59:05 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 102 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 0 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 2 successful
CapiNozzle 2017-04-25T13:59:05.862-05:00 [INFO] Send batch count=2
CapiNozzle 2017-04-25T13:59:05.862-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:59:05.864-05:00 [ERROR] Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f1/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}

CapiNozzle 2017-04-25T13:59:05.864-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err Received malformed response from tcp connection vb=200. err=malformed HTTP response "charset=ISO-8859-1\"/>", num_bytes=300, res_buf={charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /metadata%2f1/_bulk_docs. Reason:
<pre>    java.lang.RuntimeException: MapperParsingException[failed to parse]; nested: IllegalArgumentException[object field starting or ending with a [.] mak}
.
2017/04/25 13:59:05 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 202 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 303 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 307 successful
CapiNozzle 2017-04-25T13:59:05.875-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
2017/04/25 13:59:05 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 205 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 207 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:59:05 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 148 successful
StatisticsManager 2017-04-25T13:59:06.270-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
2017/04/25 13:59:06 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 45 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 49 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 157 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 151 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 54 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 159 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 52 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 158 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 160 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 156 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 51 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 53 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 259 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 253 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 258 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 255 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 260 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 252 successful
CapiNozzle 2017-04-25T13:59:06.378-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:59:06.418-05:00 [INFO] Send batch count=500
2017/04/25 13:59:06 UPR_STREAMREQ for vb 162 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 167 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 161 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 62 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 163 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 57 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 59 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 164 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 165 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 166 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 60 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 63 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 56 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 61 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 55 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 168 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 58 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 264 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 262 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 261 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 263 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 265 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 64 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 170 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 65 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 169 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 173 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 174 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 266 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 171 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 267 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 172 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 268 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 269 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 71 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 66 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 68 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 69 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 67 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 70 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 271 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 175 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 270 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 179 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 177 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 273 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 178 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 272 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 176 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 74 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 72 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 76 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 75 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 73 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 188 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 185 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 187 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 191 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 182 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 284 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 190 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 283 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 181 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 285 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 186 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 193 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 276 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 275 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 274 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 184 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 281 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 180 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 278 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 280 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 183 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 279 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 277 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 192 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 282 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 189 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 77 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 83 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 87 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 79 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 88 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 89 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 80 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 85 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 84 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 86 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 78 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 82 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 81 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 197 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 195 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 289 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 290 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 292 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 288 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 199 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 287 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 291 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 286 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 194 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 196 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 198 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 91 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 90 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 92 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 95 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 96 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 93 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 94 successful
CheckpointManager 2017-04-25T13:59:06.930-05:00 [INFO] Done with setting starting seqno for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
CheckpointManager 2017-04-25T13:59:06.930-05:00 [INFO] Done with SetVBTimestamps
2017/04/25 13:59:06 UPR_STREAMREQ for vb 296 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 295 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 297 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 298 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 299 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 293 successful
2017/04/25 13:59:06 UPR_STREAMREQ for vb 294 successful
2017/04/25 13:59:07 UPR_STREAMREQ for vb 98 successful
2017/04/25 13:59:07 UPR_STREAMREQ for vb 99 successful
2017/04/25 13:59:07 UPR_STREAMREQ for vb 97 successful
DcpNozzle 2017-04-25T13:59:07.061-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: all dcp stream are initialized.
DcpNozzle 2017-04-25T13:59:07.109-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: all dcp stream are initialized.
StatisticsManager 2017-04-25T13:59:07.229-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:59:07.245-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:59:07.269-05:00 [INFO] Retrying update docs for vb 100 for the 1th time
CapiNozzle 2017-04-25T13:59:07.617-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:59:07.651-05:00 [INFO] Retrying update docs for vb 200 for the 1th time
StatisticsManager 2017-04-25T13:59:08.182-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:59:08.386-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:59:08.401-05:00 [INFO] Retrying update docs for vb 100 for the 2th time
CapiNozzle 2017-04-25T13:59:08.865-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:59:08.883-05:00 [INFO] Retrying update docs for vb 200 for the 2th time
StatisticsManager 2017-04-25T13:59:09.281-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
CapiNozzle 2017-04-25T13:59:09.551-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:59:09.567-05:00 [INFO] Retrying update docs for vb 100 for the 3th time
CapiNozzle 2017-04-25T13:59:09.788-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:59:09.811-05:00 [INFO] Retrying update docs for vb 200 for the 3th time
StatisticsManager 2017-04-25T13:59:10.233-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=3, changes_left=315505
PipelineManager 2017-04-25T13:59:10.385-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
CapiNozzle 2017-04-25T13:59:10.708-05:00 [ERROR] batchUpdateDocs for vb 100 failed with err receieved unexpected status code, 500, from update docs request for vb 100
.
CapiNozzle 2017-04-25T13:59:10.708-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 100 after 3 retries
CapiNozzle 2017-04-25T13:59:10.708-05:00 [ERROR] Raise error condition batch update docs failed for vb 100 after 3 retries
GenericSupervisor 2017-04-25T13:59:10.708-05:00 [ERROR] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
ReplicationManager 2017-04-25T13:59:10.708-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata of type *supervisor.GenericSupervisor reported errors map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]
PipelineManager 2017-04-25T13:59:10.732-05:00 [INFO] Pipeline updater f4268e62702130298bf87f17cc481219/metadata/metadata is lauched with retry_interval=10
CapiNozzle 2017-04-25T13:59:10.732-05:00 [INFO] Send batch count=500
PipelineManager 2017-04-25T13:59:10.735-05:00 [INFO] err_list=[{"time":"2017-04-25T13:59:10.735922992-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]
PipelineManager 2017-04-25T13:59:10.736-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 1 from 0
CapiNozzle 2017-04-25T13:59:10.843-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:10.844-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:10.864-05:00 [INFO] Retrying update docs for vb 107 for the 1th time
CapiNozzle 2017-04-25T13:59:11.057-05:00 [ERROR] batchUpdateDocs for vb 200 failed with err receieved unexpected status code, 500, from update docs request for vb 200
.
CapiNozzle 2017-04-25T13:59:11.057-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 200 after 3 retries
CapiNozzle 2017-04-25T13:59:11.057-05:00 [ERROR] Raise error condition batch update docs failed for vb 200 after 3 retries
GenericSupervisor 2017-04-25T13:59:11.057-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:59:11.057-05:00 [INFO] Send batch count=500
StatisticsManager 2017-04-25T13:59:11.173-05:00 [INFO] Pipeline is no longer running, exit.
StatisticsManager 2017-04-25T13:59:11.173-05:00 [INFO] expvar=Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865 {"Errors": "[{\"time\":\"2017-04-25T13:59:10.735922992-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"": 0, "bandwidth_usage": 0, "changes_left": 315505, "data_replicated": 504, "dcp_datach_length": 532, "dcp_dispatch_time": 19, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 614, "docs_filtered": 0, "docs_opt_repd": 2, "docs_processed": 3, "docs_received_from_dcp": 157831, "docs_rep_queue": 0, "docs_written": 2, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 0, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 2, "set_failed_cr_source": 614, "set_filtered": 0, "set_received_from_dcp": 157831, "size_rep_queue": 0, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 0}, "Progress": "Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries]", "Status": "Pending"}

CapiNozzle 2017-04-25T13:59:11.202-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:11.202-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:11.242-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
CapiNozzle 2017-04-25T13:59:12.766-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:59:12.795-05:00 [INFO] Retrying update docs for vb 107 for the 2th time
CapiNozzle 2017-04-25T13:59:13.022-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:59:13.098-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
CapiNozzle 2017-04-25T13:59:14.112-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
ToplogyChangeDetector 2017-04-25T13:59:15.981-05:00 [INFO] Pipeline is no longer running, exit.
GenericSupervisor 2017-04-25T13:59:15.998-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:15.292-05:00 [INFO] Retrying update docs for vb 107 for the 3th time
GenericSupervisor 2017-04-25T13:59:16.008-05:00 [INFO] Pipeline is no longer running, exit.
CapiNozzle 2017-04-25T13:59:16.244-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
GenericSupervisor 2017-04-25T13:59:16.168-05:00 [INFO] Finish executing 0x681fb0
CapiNozzle 2017-04-25T13:59:16.878-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:59:18.379-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:59:18.429-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 300 after 3 retries
CapiNozzle 2017-04-25T13:59:18.429-05:00 [ERROR] Raise error condition batch update docs failed for vb 300 after 3 retries
GenericSupervisor 2017-04-25T13:59:18.430-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:59:18.474-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:59:18.506-05:00 [ERROR] batchUpdateDocs for vb 107 failed with err receieved unexpected status code, 500, from update docs request for vb 107
.
CapiNozzle 2017-04-25T13:59:18.506-05:00 [ERROR] Error updating docs on target. err=batch update docs failed for vb 107 after 3 retries
CapiNozzle 2017-04-25T13:59:18.506-05:00 [ERROR] Raise error condition batch update docs failed for vb 107 after 3 retries
GenericSupervisor 2017-04-25T13:59:18.506-05:00 [INFO] Received error report : map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries], but error is ignored. pipeline_state=5
CapiNozzle 2017-04-25T13:59:18.506-05:00 [INFO] Send batch count=500
CapiNozzle 2017-04-25T13:59:18.834-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:18.857-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:18.805-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
CapiNozzle 2017-04-25T13:59:18.870-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err write tcp 10.202.98.33:9091: broken pipe.
CapiNozzle 2017-04-25T13:59:18.929-05:00 [INFO] Retrying update docs for vb 4 for the 1th time
CapiNozzle 2017-04-25T13:59:18.944-05:00 [INFO] Retrying update docs for vb 201 for the 1th time
CapiNozzle 2017-04-25T13:59:19.639-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:59:19.650-05:00 [INFO] Retrying update docs for vb 4 for the 2th time
CapiNozzle 2017-04-25T13:59:19.860-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
CapiNozzle 2017-04-25T13:59:19.871-05:00 [INFO] Retrying update docs for vb 201 for the 2th time
CapiNozzle 2017-04-25T13:59:20.537-05:00 [ERROR] batchUpdateDocs for vb 4 failed with err receieved unexpected status code, 500, from update docs request for vb 4
.
CapiNozzle 2017-04-25T13:59:20.548-05:00 [INFO] Retrying update docs for vb 4 for the 3th time
PipelineManager 2017-04-25T13:59:20.737-05:00 [INFO] Try to fix Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata. Current error=capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries 
PipelineManager 2017-04-25T13:59:20.739-05:00 [INFO] Try to stop pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineManager 2017-04-25T13:59:20.740-05:00 [INFO] Try to stop the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:59:20.741-05:00 [ERROR] batchUpdateDocs for vb 201 failed with err receieved unexpected status code, 500, from update docs request for vb 201
.
GenericPipeline 2017-04-25T13:59:20.742-05:00 [INFO] Stopping pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
PipelineRuntimeCtx 2017-04-25T13:59:20.744-05:00 [INFO] Pipeline context is stopping...
CheckpointManager 2017-04-25T13:59:20.748-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:59:20.748-05:00 [INFO] Received finish signal
CheckpointManager 2017-04-25T13:59:20.749-05:00 [INFO] Exits massCheckVBOpaquesJob routine.
CheckpointManager 2017-04-25T13:59:20.749-05:00 [INFO] Exits checkpointing routine.
StatisticsManager 2017-04-25T13:59:20.750-05:00 [INFO] StatisticsManager Stopping...
StatisticsManager 2017-04-25T13:59:20.750-05:00 [INFO] StatisticsManager Stopped
ToplogyChangeDetector 2017-04-25T13:59:20.751-05:00 [INFO] ToplogyChangeDetectorSvc has stopped
GenericSupervisor 2017-04-25T13:59:20.752-05:00 [INFO] Stopping supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:59:20.752-05:00 [INFO] Supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata exited
CapiNozzle 2017-04-25T13:59:20.755-05:00 [INFO] Retrying update docs for vb 201 for the 3th time
GenericSupervisor 2017-04-25T13:59:20.754-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:20.752100051 -0500 CDT
GenericSupervisor 2017-04-25T13:59:20.760-05:00 [INFO] Stopped supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:59:20.760-05:00 [INFO] Removing child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata from supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:59:20.760-05:00 [INFO] Pipeline context for f4268e62702130298bf87f17cc481219/metadata/metadata-22514865 has been stopped
GenericPipeline 2017-04-25T13:59:20.773-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-22514865 is stopped
PipelineManager 2017-04-25T13:59:20.773-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata is stopped
PipelineManager 2017-04-25T13:59:20.773-05:00 [INFO] Replication Status=name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Pending}, errors={[{"time":"2017-04-25T13:59:10.735922992-05:00","errMsg":"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries"}]}, progress={Pipeline is stopped}

PipelineManager 2017-04-25T13:59:20.776-05:00 [INFO] Starting the pipeline f4268e62702130298bf87f17cc481219/metadata/metadata
GenericPipeline 2017-04-25T13:59:20.789-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
GenericPipeline 2017-04-25T13:59:20.794-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 for f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
GenericPipeline 2017-04-25T13:59:20.794-05:00 [INFO] Trying to stop part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
GenericPipeline 2017-04-25T13:59:20.794-05:00 [INFO] Trying to stop part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 for f4268e62702130298bf87f17cc481219/metadata/metadata-22514865
AsyncListener 2017-04-25T13:59:20.795-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:20.798-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopping processing events
AsyncListener 2017-04-25T13:59:20.799-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopping processing events
GenericPipeline 2017-04-25T13:59:20.799-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.802-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:59:20.802-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:20.802-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
DcpNozzle 2017-04-25T13:59:20.800-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopping...
CapiNozzle 2017-04-25T13:59:20.806-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1
GenericPipeline 2017-04-25T13:59:20.806-05:00 [INFO] About to call function
DcpNozzle 2017-04-25T13:59:20.806-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopping...
DcpNozzle 2017-04-25T13:59:20.805-05:00 [INFO] Closing dcp streams for vb=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
DcpNozzle 2017-04-25T13:59:20.806-05:00 [INFO] Closing dcp streams for vb=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
GenericPipeline 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:20.809-05:00 [INFO] Stop CapiNozzle capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:20.809-05:00 [INFO] closing batches ready for capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0
AsyncListener 2017-04-25T13:59:20.809-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.810-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.810-05:00 [INFO] About to call function
AsyncListener 2017-04-25T13:59:20.810-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:20.810-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:20.80653885 -0500 CDT
DcpNozzle 2017-04-25T13:59:20.810-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData exits
CapiNozzle 2017-04-25T13:59:20.810-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:20.809968333 -0500 CDT
CapiNozzle 2017-04-25T13:59:20.810-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData_batch exits
CapiNozzle 2017-04-25T13:59:20.810-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 selfMonitor routine exits
CapiNozzle 2017-04-25T13:59:20.811-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData_batch exits
CapiNozzle 2017-04-25T13:59:20.811-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 selfMonitor routine exits
AsyncListener 2017-04-25T13:59:20.811-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.813-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 stopped processing events
CapiNozzle 2017-04-25T13:59:20.813-05:00 [INFO] releasing capi client
CapiNozzle 2017-04-25T13:59:20.814-05:00 [INFO] releasing capi client
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.814-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 stopped processing events
GenericPipeline 2017-04-25T13:59:20.814-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:20.815-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is stopped
GenericPipeline 2017-04-25T13:59:20.815-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:20.815-05:00 [INFO] part capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is stopped
DcpNozzle 2017-04-25T13:59:20.817-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:59:20.817-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:20.817797231 -0500 CDT
DcpNozzle 2017-04-25T13:59:20.817-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
GenericPipeline 2017-04-25T13:59:20.817-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:20.817-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is stopped
2017/04/25 13:59:20 Stream Closed for vb 0 StreamEnd simulated
2017/04/25 13:59:20 Stream Closed for vb 1 StreamEnd simulated
2017/04/25 13:59:20 Stream Closed for vb 2 StreamEnd simulated
DcpNozzle 2017-04-25T13:59:20.818-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData exits
AsyncListener 2017-04-25T13:59:20.818-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.818-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:20.818-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.818-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:20.818-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.818-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 stopped processing events
DcpNozzle 2017-04-25T13:59:20.819-05:00 [INFO] Ask uprfeed to close
DcpNozzle 2017-04-25T13:59:20.819-05:00 [INFO] server is stopped per request sent at 2017-04-25 13:59:20.819843557 -0500 CDT
DcpNozzle 2017-04-25T13:59:20.819-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
GenericPipeline 2017-04-25T13:59:20.819-05:00 [INFO] Finish executing 0x65e930
GenericPipeline 2017-04-25T13:59:20.819-05:00 [INFO] part dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is stopped
AsyncListener 2017-04-25T13:59:20.819-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 stopped processing events
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 stopped processing events
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] Finish executing 0x793f70
AsyncListener 2017-04-25T13:59:20.820-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 stopped processing events
2017/04/25 13:59:20 Stream Closed for vb 171 StreamEnd simulated
XDCRFactory 2017-04-25T13:59:20.963-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfDcpNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:59:20.967-05:00 [INFO] Constructed 2 source nozzles for 341 vbs on 10.202.98.30:11210
XDCRFactory 2017-04-25T13:59:20.967-05:00 [INFO] kv_vb_map=map[10.202.98.30:11210:[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]]
XDCRFactory 2017-04-25T13:59:21.091-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, numOfOutNozzles=2, numOfVbs=341, load_distribution=[[0 171] [171 341]]
XDCRFactory 2017-04-25T13:59:21.095-05:00 [INFO] Constructed 2 outgoing nozzles
XDCRRouter 2017-04-25T13:59:21.102-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:59:21.102-05:00 [INFO] Constructed router
XDCRRouter 2017-04-25T13:59:21.102-05:00 [INFO] Router created with 1 downstream parts 
XDCRFactory 2017-04-25T13:59:21.102-05:00 [INFO] Constructed router
XDCRFactory 2017-04-25T13:59:21.102-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_sources=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
XDCRFactory 2017-04-25T13:59:21.104-05:00 [INFO] topic=f4268e62702130298bf87f17cc481219/metadata/metadata, num_of_targets=2, num_of_listeners=2, load_distribution=[[0 1] [1 2]]
ThroughSeqnoTrackerSvc 2017-04-25T13:59:21.112-05:00 [INFO] Attach through seqno tracker with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
GenericSupervisor 2017-04-25T13:59:21.118-05:00 [INFO] Adding child PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata to supervisor PipelineMasterSupervisor
PipelineRuntimeCtx 2017-04-25T13:59:21.118-05:00 [INFO] Try to attach PipelineSupervisor to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
GenericSupervisor 2017-04-25T13:59:21.119-05:00 [INFO] Attaching pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918 to supervior service PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:21.119-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:21.119-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:21.119-05:00 [INFO] Adding child dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:21.119-05:00 [INFO] Adding child capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 to supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata
PipelineRuntimeCtx 2017-04-25T13:59:21.126-05:00 [INFO] Try to attach CheckpointManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
CheckpointManager 2017-04-25T13:59:21.126-05:00 [INFO] Attach checkpoint manager with pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
CapiUtils 2017-04-25T13:59:21.193-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:59:21.193-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:59:21.194-05:00 [INFO] Remote bucket elasticsearch - metadata supporting xdcrcheckpoing is false
PipelineRuntimeCtx 2017-04-25T13:59:21.198-05:00 [INFO] Try to attach StatisticsManager to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
StatisticsManager 2017-04-25T13:59:21.207-05:00 [INFO] StatisticsManager is started for pipeline 0x6828a0
PipelineRuntimeCtx 2017-04-25T13:59:21.208-05:00 [INFO] Try to attach TopologyChangeDetectSvc to pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
XDCRFactory 2017-04-25T13:59:21.208-05:00 [INFO] XDCR pipeline constructed
PipelineManager 2017-04-25T13:59:21.209-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918 is constructed, start it
GenericPipeline 2017-04-25T13:59:21.211-05:00 [INFO] Starting pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918
 ------------f4268e62702130298bf87f17cc481219/metadata/metadata---------------
	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0:{vbList=[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[89 151 107 120 53 91 98 131 145 23 74 96 9 33 85 11 72 51 80 102 112 7 16 19 49 149 165 144 40 59 92 122 54 77 84 62 128 163 90 115 118 162 167 65 142 148 31 69 104 87 105 14 60 68 78 32 100 160 44 64 138 73 17 28 130 18 42 110 8 26 117 141 143 154 161 2 71 136 166 168 170 13 56 93 132 135 30 43 58 152 39 127 38 124 133 1 15 94 12 134 169 20 101 153 150 157 6 79 116 129 108 159 21 25 27 63 4 67 83 158 125 22 34 52 66 139 37 137 41 46 113 99 10 82 106 146 0 3 35 76 70 86 155 156 75 111 126 88 45 164 24 48 57 97 119 55 61 81 103 123 140 29 50 114 5 36 95 109 47 121 147] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[212 301 219 198 302 332 208 231 242 280 311 333 186 264 339 245 240 286 310 194 229 257 330 292 184 218 272 290 238 233 225 278 288 299 317 193 248 277 318 256 224 275 284 306 180 326 214 203 204 293 294 241 216 297 303 239 196 276 322 323 234 295 308 316 172 321 260 191 300 305 328 175 199 211 327 336 187 201 246 266 285 320 249 267 281 304 232 243 331 217 312 230 319 338 207 222 325 205 200 202 255 283 296 171 215 178 273 265 291 307 206 235 262 268 274 329 188 226 244 282 185 190 177 221 228 253 252 195 314 270 189 210 176 183 309 174 247 254 279 324 337 220 334 340 192 227 250 271 209 237 197 261 298 213 223 263 269 335 258 289 173 236 287 181 313 182 259 315 251 179]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0

	dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1:{vbList=[171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]}
		Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 :{
routingMap=map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:[135 30 43 58 152 39 127 38 124 133 1 15 94 12 134 169 20 101 153 157 6 79 116 129 150 159 21 25 27 63 108 4 67 83 158 22 34 52 66 125 139 37 137 41 46 113 99 10 106 146 0 3 35 76 82 70 86 155 156 75 111 126 88 45 24 48 57 97 164 55 61 81 103 119 123 140 29 50 114 5 36 95 109 47 121 147 89 151 107 120 53 91 98 131 145 23 74 96 9 33 85 11 72 80 102 112 7 16 19 49 51 149 165 40 59 92 122 144 54 77 84 62 128 163 90 115 118 162 167 65 142 148 31 69 104 105 14 60 68 78 87 32 100 160 44 64 138 73 17 28 130 18 42 110 8 26 117 141 143 154 161 2 71 136 166 170 13 56 93 132 168] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:[187 246 266 285 201 320 267 281 304 232 243 249 217 331 312 319 338 207 230 325 205 222 200 202 255 283 296 171 215 178 273 291 307 206 235 262 265 274 329 188 226 244 268 185 190 282 177 221 228 253 252 195 314 270 189 210 176 183 309 247 254 279 174 324 220 337 340 192 227 250 271 334 209 237 197 261 298 213 223 263 269 335 289 173 236 258 287 181 313 259 315 182 179 251 212 301 219 198 302 332 231 242 280 208 311 333 186 264 339 245 286 310 194 240 229 257 330 292 184 218 272 290 238 225 233 288 299 317 193 248 278 318 256 277 224 275 284 180 306 214 326 203 204 293 241 294 216 303 239 297 196 276 323 234 322 295 308 316 321 172 260 300 305 328 191 175 199 211 327 336]]}
			capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1


-----------------------------
 
 settings = map[replication_type:capi filter_expression: checkpoint_interval:1800 stats_interval:1000 log_level:Info active:true worker_batch_size:500 doc_batch_size_kb:2048 failure_restart_interval:10 optimistic_replication_threshold:256 source_nozzle_per_node:2 target_nozzle_per_node:2]
CheckpointManager 2017-04-25T13:59:21.213-05:00 [INFO] Set start seqnos for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918...
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 started processing events
AsyncListener 2017-04-25T13:59:21.256-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 started processing events
GenericSupervisor 2017-04-25T13:59:21.257-05:00 [INFO] Starting supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
GenericSupervisor 2017-04-25T13:59:21.258-05:00 [INFO] Started supervisor PipelineSupervisor_f4268e62702130298bf87f17cc481219/metadata/metadata.
PipelineRuntimeCtx 2017-04-25T13:59:21.258-05:00 [INFO] Service PipelineSupervisor has been started
GenericSupervisor 2017-04-25T13:59:21.258-05:00 [INFO] monitorPipelineHealth started
CheckpointManager 2017-04-25T13:59:21.259-05:00 [INFO] CheckpointManager starting with ckpt_interval=1800 s
CheckpointManager 2017-04-25T13:59:21.262-05:00 [INFO] Checkpointing starts in 1801.028 sec
PipelineRuntimeCtx 2017-04-25T13:59:21.262-05:00 [INFO] Service CheckpointManager has been started
CheckpointManager 2017-04-25T13:59:21.262-05:00 [INFO] checkpointing rountine started
StatisticsManager 2017-04-25T13:59:21.263-05:00 [INFO] StatisticsManager Starting...
PipelineRuntimeCtx 2017-04-25T13:59:21.272-05:00 [INFO] Service StatisticsManager has been started
ToplogyChangeDetector 2017-04-25T13:59:21.273-05:00 [INFO] ToplogyChangeDetectorSvc has started
PipelineRuntimeCtx 2017-04-25T13:59:21.273-05:00 [INFO] Service TopologyChangeDetectSvc has been started
StatisticsManager 2017-04-25T13:59:21.274-05:00 [INFO] updateStats started
CapiUtils 2017-04-25T13:59:21.275-05:00 [INFO] serverCouchApiBaseMap = map[10.202.98.33:9091:http://10.202.98.33:9091/metadata]
CheckpointManager 2017-04-25T13:59:21.275-05:00 [INFO] remoteBucket.MemcachedAddrRestAddrMap=map[10.202.98.33:9091:10.202.98.33:9091]
CheckpointManager 2017-04-25T13:59:21.276-05:00 [INFO] Getting checkpoint for f4268e62702130298bf87f17cc481219/metadata/metadata
CapiNozzle 2017-04-25T13:59:21.288-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 starting ....
CheckpointManager 2017-04-25T13:59:21.291-05:00 [INFO] Found 0 checkpoit document for replication f4268e62702130298bf87f17cc481219/metadata/metadata
ToplogyChangeDetector 2017-04-25T13:59:21.320-05:00 [INFO] validateSourceTopology...
CapiNozzle 2017-04-25T13:59:21.337-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 initialized
CapiNozzle 2017-04-25T13:59:21.337-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 is started successfully
CapiNozzle 2017-04-25T13:59:21.337-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 processData starts..........
DcpNozzle 2017-04-25T13:59:21.352-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
DcpNozzle 2017-04-25T13:59:21.352-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 starting ....
CheckpointManager 2017-04-25T13:59:21.384-05:00 [INFO] Got the bucket metadata for ckmgr
StatisticsManager 2017-04-25T13:59:21.420-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
DcpNozzle 2017-04-25T13:59:21.488-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is initialized
DcpNozzle 2017-04-25T13:59:21.488-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:59:21.488-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 is started
DcpNozzle 2017-04-25T13:59:21.509-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 processData starts..........
DcpNozzle 2017-04-25T13:59:21.511-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1: startUprStreams for [171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]...
CapiNozzle 2017-04-25T13:59:21.515-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 starting ....
CapiNozzle 2017-04-25T13:59:21.570-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 initialized
CapiNozzle 2017-04-25T13:59:21.571-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 is started successfully
CapiNozzle 2017-04-25T13:59:21.573-05:00 [INFO] capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 processData starts..........
DcpNozzle 2017-04-25T13:59:21.584-05:00 [INFO] Dcp nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
DcpNozzle 2017-04-25T13:59:21.584-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 starting ....
CheckpointManager 2017-04-25T13:59:21.851-05:00 [INFO] StartSeqnoGetter 2 is started to do _pre_prelicate for vb [200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299]
CheckpointManager 2017-04-25T13:59:21.822-05:00 [INFO] StartSeqnoGetter 1 is started to do _pre_prelicate for vb [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199]
CheckpointManager 2017-04-25T13:59:21.874-05:00 [INFO] StartSeqnoGetter 3 is started to do _pre_prelicate for vb [300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340]
CheckpointManager 2017-04-25T13:59:21.772-05:00 [INFO] StartSeqnoGetter 0 is started to do _pre_prelicate for vb [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99]
DcpNozzle 2017-04-25T13:59:22.295-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is initialized
DcpNozzle 2017-04-25T13:59:22.840-05:00 [INFO] Dcp nozzle is started
GenericPipeline 2017-04-25T13:59:24.391-05:00 [INFO] Incoming nozzle dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 is started
GenericPipeline 2017-04-25T13:59:24.423-05:00 [INFO] All parts has been started
DcpNozzle 2017-04-25T13:59:24.216-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 processData starts..........
DcpNozzle 2017-04-25T13:59:23.840-05:00 [INFO] dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0: startUprStreams for [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170]...
GenericPipeline 2017-04-25T13:59:24.809-05:00 [INFO] -----------Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918 is started----------
PipelineManager 2017-04-25T13:59:24.813-05:00 [INFO] Replication f4268e62702130298bf87f17cc481219/metadata/metadata has been updated. Back to business
PipelineManager 2017-04-25T13:59:24.813-05:00 [INFO] Pipeline f4268e62702130298bf87f17cc481219/metadata/metadata has been updated successfully
PipelineManager 2017-04-25T13:59:24.816-05:00 [INFO] Updater f4268e62702130298bf87f17cc481219/metadata/metadata moved to 2 from 1
2017/04/25 13:59:25 UPR_STREAMREQ for vb 0 successful
StatisticsManager 2017-04-25T13:59:25.057-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:59:25.083-05:00 [INFO] Send batch count=108
2017/04/25 13:59:25 UPR_STREAMREQ for vb 101 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 100 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 300 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 200 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 201 successful
StatisticsManager 2017-04-25T13:59:25.194-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=0, changes_left=315508
CapiNozzle 2017-04-25T13:59:25.196-05:00 [INFO] Send batch count=150
2017/04/25 13:59:25 UPR_STREAMREQ for vb 1 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 102 successful
CapiNozzle 2017-04-25T13:59:25.352-05:00 [ERROR] Received error when writing boby part. err=write tcp 10.202.98.33:9091: broken pipe
2017/04/25 13:59:25 UPR_STREAMREQ for vb 301 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 203 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 302 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 202 successful
CapiNozzle 2017-04-25T13:59:25.363-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err write tcp 10.202.98.33:9091: broken pipe.
2017/04/25 13:59:25 UPR_STREAMREQ for vb 2 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 103 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 303 successful
CapiNozzle 2017-04-25T13:59:25.417-05:00 [INFO] Retrying update docs for vb 0 for the 1th time
CapiNozzle 2017-04-25T13:59:25.454-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:59:25 UPR_STREAMREQ for vb 3 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 104 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 4 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 206 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 204 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 305 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 304 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 205 successful
StatisticsManager 2017-04-25T13:59:25.506-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
CapiNozzle 2017-04-25T13:59:25.506-05:00 [INFO] Retrying update docs for vb 300 for the 1th time
2017/04/25 13:59:25 UPR_STREAMREQ for vb 105 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 5 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 306 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 207 successful
PipelineManager 2017-04-25T13:59:25.581-05:00 [INFO] Replication Status = map[f4268e62702130298bf87f17cc481219/metadata/metadata:name={f4268e62702130298bf87f17cc481219/metadata/metadata}, status={Replicating}, errors={[]}, progress={Pipeline is running}
]
2017/04/25 13:59:25 UPR_STREAMREQ for vb 10 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 111 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 112 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 6 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 7 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 109 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 113 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 107 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 108 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 9 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 106 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 110 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 8 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 311 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 314 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 310 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 211 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 312 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 309 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 208 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 209 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 307 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 308 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 210 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 313 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 14 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 117 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 114 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 13 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 11 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 12 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 119 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 115 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 118 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 116 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 15 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 16 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 213 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 212 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 318 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 214 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 316 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 319 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 215 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 315 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 317 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 120 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 18 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 129 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 123 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 127 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 19 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 21 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 128 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 25 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 17 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 124 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 121 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 22 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 24 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 26 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 125 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 122 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 126 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 23 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 20 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 130 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 216 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 324 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 323 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 225 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 327 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 227 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 224 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 320 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 223 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 217 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 222 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 220 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 219 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 221 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 321 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 322 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 325 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 226 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 326 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 328 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 218 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 139 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 33 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 142 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 28 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 38 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 136 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 32 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 131 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 29 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 132 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 134 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 137 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 37 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 30 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 41 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 40 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 138 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 39 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 133 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 141 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 140 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 27 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 34 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 35 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 36 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 31 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 135 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 240 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 332 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 337 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 340 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 330 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 237 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 336 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 235 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 335 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 228 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 242 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 241 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 329 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 338 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 233 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 234 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 243 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 331 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 239 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 231 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 238 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 230 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 229 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 334 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 236 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 232 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 333 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 339 successful
2017/04/25 13:59:25 UPR_STREAMREQ for vb 244 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 250 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 245 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 249 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 248 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 246 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 247 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 251 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 48 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 149 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 46 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 146 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 148 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 147 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 44 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 145 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 42 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 144 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 43 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 47 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 143 successful
2017/04/25 13:59:26 UPR_STREAMREQ for vb 45 successful
CapiNozzle 2017-04-25T13:59:27.474-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
CapiNozzle 2017-04-25T13:59:27.457-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
CapiNozzle 2017-04-25T13:59:27.645-05:00 [INFO] Retrying update docs for vb 0 for the 2th time
2017/04/25 13:59:27 UPR_STREAMREQ for vb 252 successful
2017/04/25 13:59:27 UPR_STREAMREQ for vb 150 successful
2017/04/25 13:59:27 UPR_STREAMREQ for vb 49 successful
CapiNozzle 2017-04-25T13:59:28.358-05:00 [INFO] Retrying update docs for vb 300 for the 2th time
GenericSupervisor 2017-04-25T13:59:32.222-05:00 [INFO] About to call function
ToplogyChangeDetector 2017-04-25T13:59:32.200-05:00 [INFO] validateSourceTopology...
GenericSupervisor 2017-04-25T13:59:34.510-05:00 [INFO] Executing 0x681fb0 timed out
StatisticsManager 2017-04-25T13:59:35.061-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
GenericSupervisor 2017-04-25T13:59:34.459-05:00 [ERROR] Heartbeat timeout in supervisor ReplicationManagerSupervisor! not_yet_resp_count=1
GenericSupervisor 2017-04-25T13:59:35.142-05:00 [INFO] Child AdminportSupervisor of supervisor ReplicationManagerSupervisor missed 1 consecutive heart beats
GenericSupervisor 2017-04-25T13:59:35.051-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:59:35.172-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
GenericSupervisor 2017-04-25T13:59:35.210-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
GenericSupervisor 2017-04-25T13:59:35.256-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:59:37 UPR_STREAMREQ for vb 151 successful
GenericSupervisor 2017-04-25T13:59:37.254-05:00 [INFO] Send heartbeat failed for AdminportSupervisor, err=Last heart beat msg has not been processed
2017/04/25 13:59:37 UPR_STREAMREQ for vb 50 successful
2017/04/25 13:59:37 UPR_STREAMREQ for vb 253 successful
StatisticsManager 2017-04-25T13:59:37.698-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:59:10.735922992-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 301, "dcp_dispatch_time": 142, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 110, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 25293, "docs_rep_queue": 22938, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 20768, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 110, "set_filtered": 0, "set_received_from_dcp": 25293, "size_rep_queue": 30621332, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 89}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 40, "docs_opt_repd": 0, "docs_rep_queue": 17312, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 40, "size_rep_queue": 22645819, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 86, "max": 124, "mean": 123.62790697674419, "min": 123}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 70, "docs_opt_repd": 0, "docs_rep_queue": 18914, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 70, "size_rep_queue": 24037330, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 134, "max": 67, "mean": 66.8955223880597, "min": 66}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 540, "dcp_dispatch_time": {"count": 11994, "max": 7412, "mean": 28.083, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 17475, "expiry_received_from_dcp": 0, "set_received_from_dcp": 17475}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 112, "dcp_dispatch_time": {"count": 13299, "max": 221867, "mean": 245.468, "min": 5}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 19729, "expiry_received_from_dcp": 0, "set_received_from_dcp": 19729}}

StatisticsManager 2017-04-25T13:59:37.708-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 17475 items, sent 108 items
StatisticsManager 2017-04-25T13:59:37.708-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 19949 items, sent 150 items
StatisticsManager 2017-04-25T13:59:38.337-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:17895]
2017/04/25 13:59:39 UPR_STREAMREQ for vb 51 successful
StatisticsManager 2017-04-25T13:59:39.434-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 18258 items, sent 18257 items. streams inactive: [162 79 81 157 170 66 60 70 73 76 93 156 159 80 58 154 95 98 51 55 62 71 90 152 72 68 75 164 56 78 99 84 169 53 52 77 85 82 91 69 74 163 54 57 67 96 158 160 167 59 65 153 166 61 63 161 97 83 168 165 87 89 64 88 94 155 92 86]
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:21396]
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 21396 items, sent 21395 items. streams inactive: [259 278 178 262 281 179 180 274 279 264 286 296 299 187 199 181 289 192 186 191 257 268 295 195 270 290 190 196 183 280 172 175 272 298 188 293 266 260 177 254 261 288 198 292 273 282 193 197 284 176 263 277 291 182 194 294 297 256 258 287 184 189 276 283 269 275 171 271 255 265 267 285 174 185 173]
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:39.582-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
CapiNozzle 2017-04-25T13:59:39.647-05:00 [ERROR] batchUpdateDocs for vb 300 failed with err receieved unexpected status code, 500, from update docs request for vb 300
.
2017/04/25 13:59:39 UPR_STREAMREQ for vb 254 successful
2017/04/25 13:59:40 UPR_STREAMREQ for vb 152 successful
2017/04/25 13:59:41 UPR_STREAMREQ for vb 52 successful
GenericSupervisor 2017-04-25T13:59:41.379-05:00 [INFO] About to call function
CapiNozzle 2017-04-25T13:59:41.346-05:00 [INFO] Retrying update docs for vb 300 for the 3th time
CapiNozzle 2017-04-25T13:59:41.494-05:00 [ERROR] batchUpdateDocs for vb 0 failed with err receieved unexpected status code, 500, from update docs request for vb 0
.
StatisticsManager 2017-04-25T13:59:41.532-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:59:41 UPR_STREAMREQ for vb 255 successful
CapiNozzle 2017-04-25T13:59:42.610-05:00 [INFO] Retrying update docs for vb 0 for the 3th time
2017/04/25 13:59:42 UPR_STREAMREQ for vb 153 successful
2017/04/25 13:59:42 UPR_STREAMREQ for vb 53 successful
GenericSupervisor 2017-04-25T13:59:42.595-05:00 [INFO] Executing 0x681fb0 timed out
GenericSupervisor 2017-04-25T13:59:42.748-05:00 [INFO] ****************************
GenericSupervisor 2017-04-25T13:59:42.751-05:00 [INFO] Received timeout error when checking pipeline health. topic=f4268e62702130298bf87f17cc481219/metadata/metadata
2017/04/25 13:59:43 UPR_STREAMREQ for vb 256 successful
2017/04/25 13:59:43 UPR_STREAMREQ for vb 154 successful
2017/04/25 13:59:43 UPR_STREAMREQ for vb 155 successful
2017/04/25 13:59:43 UPR_STREAMREQ for vb 257 successful
2017/04/25 13:59:43 UPR_STREAMREQ for vb 54 successful
StatisticsManager 2017-04-25T13:59:43.678-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata total_docs=315508, docs_processed=4, changes_left=315504
2017/04/25 13:59:43 UPR_STREAMREQ for vb 258 successful
ToplogyChangeDetector 2017-04-25T13:59:43.597-05:00 [INFO] validateSourceTopology...
StatisticsManager 2017-04-25T13:59:43.819-05:00 [INFO] Stats for pipeline f4268e62702130298bf87f17cc481219/metadata/metadata-102890918 {"CkptMgr": {"num_checkpoints": 0, "num_failedckpts": 0, "time_committing": {"count": 0, "max": 0, "mean": 0, "min": 0}}, "Errors": "[{\"time\":\"2017-04-25T13:59:10.735922992-05:00\",\"errMsg\":\"capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:batch update docs failed for vb 100 after 3 retries\"}]", "Overview": {"bandwidth_usage": 0, "changes_left": 315504, "data_replicated": 0, "dcp_datach_length": 821, "dcp_dispatch_time": 542, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "deletion_filtered": 0, "deletion_received_from_dcp": 0, "docs_checked": 0, "docs_failed_cr_source": 110, "docs_filtered": 0, "docs_opt_repd": 0, "docs_processed": 4, "docs_received_from_dcp": 52680, "docs_rep_queue": 49877, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "expiry_filtered": 0, "expiry_received_from_dcp": 0, "num_checkpoints": 0, "num_failedckpts": 0, "rate_doc_checks": 0, "rate_doc_opt_repd": 0, "rate_received_from_dcp": 11826, "rate_replicated": 0, "resp_wait_time": 0, "set_docs_written": 0, "set_failed_cr_source": 110, "set_filtered": 0, "set_received_from_dcp": 52680, "size_rep_queue": 62976814, "time_committing": 0, "wtavg_docs_latency": 0, "wtavg_meta_latency": 89}, "Progress": "Pipeline is running", "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"deletion_filtered": 0, "docs_filtered": 0, "expiry_filtered": 0, "set_filtered": 0}, "Status": "Replicating", "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 40, "docs_opt_repd": 0, "docs_rep_queue": 33279, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 40, "size_rep_queue": 42576053, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 86, "max": 124, "mean": 123.62790697674419, "min": 123}}, "capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1": {"data_replicated": 0, "deletion_docs_written": 0, "deletion_failed_cr_source": 0, "docs_failed_cr_source": 70, "docs_opt_repd": 0, "docs_rep_queue": 27644, "docs_written": 0, "expiry_docs_written": 0, "expiry_failed_cr_source": 0, "resp_wait_time": {"count": 0, "max": 0, "mean": 0, "min": 0}, "set_docs_written": 0, "set_failed_cr_source": 70, "size_rep_queue": 33693444, "wtavg_docs_latency": {"count": 0, "max": 0, "mean": 0, "min": 0}, "wtavg_meta_latency": {"count": 134, "max": 67, "mean": 66.8955223880597, "min": 66}}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0": {"dcp_datach_length": 270, "dcp_dispatch_time": {"count": 27348, "max": 550436, "mean": 580.992, "min": 9}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 36243, "expiry_received_from_dcp": 0, "set_received_from_dcp": 36243}, "dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1": {"dcp_datach_length": 551, "dcp_dispatch_time": {"count": 25245, "max": 333006, "mean": 500.05, "min": 9}, "deletion_received_from_dcp": 0, "docs_received_from_dcp": 32252, "expiry_received_from_dcp": 0, "set_received_from_dcp": 32252}}

StatisticsManager 2017-04-25T13:59:43.820-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0 received 36243 items, sent 108 items
StatisticsManager 2017-04-25T13:59:43.820-05:00 [INFO] Capi capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1 received 32252 items, sent 150 items
StatisticsManager 2017-04-25T13:59:43.824-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_0:36243]
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_0 received 36243 items, sent 36243 items. streams inactive: [170 81 157 60 66 76 93 156 70 73 159 80 58 95 98 55 62 71 72 90 68 164 75 99 56 78 169 84 77 85 82 91 69 74 163 57 67 96 158 167 160 59 65 166 161 61 63 83 97 168 165 89 87 64 88 94 86 92 162 79]
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] Rounter Router_dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 = map[capi_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.33:9091_1:32252]
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] Dcp dcp_f4268e62702130298bf87f17cc481219/metadata/metadata_10.202.98.30:11210_1 received 32252 items, sent 32252 items. streams inactive: [264 274 279 187 199 286 296 299 181 289 186 191 192 195 268 295 190 196 270 290 172 175 183 280 188 272 298 293 266 177 260 261 288 198 292 273 193 197 282 284 176 277 291 182 194 263 294 297 287 184 189 276 269 283 171 275 271 174 185 265 267 285 173 178 259 278 262 179 180 281]
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataReceivedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_GetMetaReceivedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_0 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFailedCREventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataSentEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataProcessedEventListener_1 chan size =0 
StatisticsManager 2017-04-25T13:59:43.825-05:00 [INFO] f4268e62702130298bf87f17cc481219/metadata/metadata_DataFilteredEventListener_1 chan size =0 
2017/04/25 13:59:43 UPR_STREAMREQ for vb 259 successful